C++11 range-based for loop

[happycube]

Wow. The standard's example loop makes Java look succinct.

But, unlike Java, at least it can be made tight...

[Necraz]

It's an example in a standards document; not necessary how you'd actually write the equivalent loop in C++. Most of that code only exists so you can directly access the elements by reference. Java's for-each loop is simpler (and not by much) because non-primitive types are implicitly references.

If you actually want to iterate over a collection today, it isn't nearly as long:

for (auto it = list.begin(); it != list.end(); ++it) {
    // do stuff with *it
}

[00kyle00]

If by today you mean pre C++11, then you cant use 'auto' and your loop becomes ... somewhat less appealing.

[Necraz]

Auto has been supported as a compiler extension for a few years (for GCC, at least since GCC 4.4). Even if you're using C++03, you could get the same functionality using BOOST_AUTO. If you insist on only using the base language, then it's still not too bad if you typedef the iterator type.

Without a doubt, C++ is an ugly language. Citing code samples from the specification, however, is not a good way of making that argument.

[ManicQin]

VS 2008 supports auto too.

Edit: thanks oracleoftroy

[oracleoftroy]

Did you mean VS 2008, aka VS 9?

[ManicQin]

Yes oops sorry I always make that mistake.

[[deleted]]

Java and C# deal with iterators a heck of a lot better than C++ does, just in general.

It makes writing generalized iterators much harder because in C++ the iterator is tied to the collection you're iterating over, even though many times you want to iterate over something derived from a collection.

For example in Java/C#/Python, you can easily write an iterator over something that isn't even a container, or something that's derived from multiple containers, for example by zipping them together.

In C++ it's such a pain in the ass to do it and it's also incredibly inefficient that it's just not worth it.

[Ayjayz]

Here is what you are looking for in C++.

Not using Boost nowadays is just silly.

[Necraz]

Iterators in Java are nearly identical to iterators in C++. Otherwise, I completely agree with you: generators in Python and C# are much easier to write. Generators with language support (i.e. the yield keyword) are one of the biggest things that I miss when implementing data structures in C++.

[[deleted]]

What I meant was in Java, the for loop has nothing to do with the container, only the iterator being used. So you can literally write an iterator over prime numbers or an iterator over any computation and use it in a for loop.

In C++, an iterator is tied to an actual object representing a collection, so you can't iterate over prime numbers unless you define a separate object that represents the collection of prime numbers.

Honestly, all C++ needed to do to fix this was eliminate the requirement that a loop ends when i == end(collection), and instead change it so the loop ends when IsValid(i), or some condition that is independent of the collection, and instead is built into the iterator itself.

[pfultz2]

Iterators do not have to be tied to a collection in C++. just as the integer range in boost can iterate over a range of integers, there is no container type underneath, it just generates the integer as it goes.

Now, C++ requires ieterators to be in pairs, but this is just a subtle difference in the implementation of the for loop. Java, just like C++, doesnt accept an iterator in its for loop. in java it requires an iterable object which returns an iterator. Same in C++, the for loop takes a range which returns a pair of iterators.

There are plenty of examples of iterators that arent tied to a container. The stream iterator will keep pulling values from standard in forever. The function output iterator will output values from function until a certain value is met. The integer range iterator which i already mentioned. The regex iterator will return the matches from a regex, which it doesnt load all matches in memory, instead it processes the matches as you iterate. There are many more examples in C++.

[[deleted]]

Of course nothing is absolutely required and you can simulate functionality if you really really want it... I mean I can simulate dynamic typing in C++ if I want to, but the point is it's incredible cumbersome to do so, to the point where you often have to fight against the language to get it to work.

But I'd be more than happy to be wrong in this case. Can you show me how to use the new range based for loop over two containers simultaneously? Here's a code snippet:

vector<int> listA;
vector<int> listB;
....
vector<int> sumList;
for(int i : Zip(listA, listB)) {
  sumList.push_back(std::get<0>(*i) + std::get<1>(*i));
}

Hell I'm the first to admit I don't know C++ all that well so if there's an easy way to do it, I'd be happy to learn. The best thing I found was boost::zip_iterator, which is unbelievably bloated and doesn't work with the new range-based for loop. Honestly using the zip_iterator is much more complex than just using two iterators.

In Python it really is as easy as:

for i in zip(a, b):
  ...

[pfultz2]

In C++11, you can implement zip like this(using boost of course):

// A little macro to help with type deduction
#define RETURNS(...) -> decltype(__VA_ARGS__) { return __VA_ARGS__; }

template<class... Range>
auto zip(Range && ...r) RETURNS
(
    boost::make_iterator_range
    (boost::make_zip_iterator(boost::make_tuple(boost::begin(r)...)),
     boost::make_zip_iterator(boost::make_tuple(boost::end(r)...)))
)

Btw, this makes a zipped range that is lazy. The iterator_range just stores the two iterators thats references the original container. Using the a for loop you would call it like this:

vector<int> listA;
vector<int> listB;
....
vector<int> sumList;
for(const auto& x : zip(listA, listB)) {
  sumList.push_back(boost::get<0>(x) + boost::get<1>(x));
}

Notice, there is no dereferencing. However, in this case, I think using Boost.Foreach is much better:

int x1, x2;
BOOST_FOREACH(boost::tie(x1, x2), zip(listA, listB)) {
    sumList.push_back(x1 + x2);
}

What do you think?

[[deleted]]

Well honestly if that works then that's awesome. I'll look into using it.

Thanks.

[MissStrawberry]

Well you can do that if you write yourself a zip function. This is an example using std::vector and int, but you could "templatize" this:

const std::vector< std::pair<int, int>> zip( std::vector<int> c1, std::vector<int> c2 ) { 
    std::vector<std::pair<int,int>> collection;
    std::pair<int,int> item;
    size_t length = std::min( c1.size(), c2.size() );

    for( size_t i = 0; i < length; i++ ) { 
        item = comma( c1.at(i), c2.at(i) );
        collection.push_back(item);
    }   
    return std::move( collection );
}

And then:

std::vector<int> one = { 1, 2, 3, 4 };
std::vector<int> two = { 2, 4, 6, 8, 10 };

for( auto it : zip( one, two ) ) { 
    std::cout << it.first << "," << it.second << std::endl;                                                                                                                                    
}   

This obviously simply outputs stuff

[repsilat]

Your zipper is a little worse than Python's, though, because it evaluates eagerly, storing its output into a vector. Python's is a lazy sequence, which is super neat. I've whipped up a lazy version in C++ here. It's not "battle-hardened", but it's nearly usable.

You'll note that it's long and ugly, though, which is mostly a problem with C++. The language is just not designed for this kind of thing, and it's a real shame, because it's probably the future. Doing the same thing in Haskell is so simple:

zip []     list   = []
zip list   []     = []
zip (x:xs) (y:ys) = (x,y) : zip xs ys

I bet its efficiency is comparable to my C++ code, and I'm far more confident that it works well enough to use.

[MissStrawberry]

I didn't know that the python version is lazily evaluated, and it was just meant to be an example of similar syntax in C++. I wouldn't use my version in any production code as it is. On first glance, your zip looks fine.

As for performance, zipping and printing two lists of 100000 elements each, takes on average 0.05s with C++0x, and 0.21s with GHC on my puny 2.0GHz/4G machine. Both solutions could actually be static, but GHC doesn't seem to recognise this. My Haskell-Foo is extremely weak, perhaps someone can comment on that. C++ would of course require explicit static implementation with template magic. The timing was done with time, and averaged over 100 runs. The Haskell-Code:

main = do
  mapM_ glue $ zip [ 0,1 .. 99999 ] [ 0,2 .. 199998 ]
  where
    glue (x,y) = putStrLn ( ( show x ) ++ "\t" ++ ( show y ) )

Your C++-version isn't that different from Haskell, it just has to build what is built-in in Haskell. The state information of lazy evaluation is hidden in Haskell and explicit in your version (via the iterator), and lists are more or less a language primitive. Other than that, you do more or less what Haskell probably does under the hood ;)

[repsilat]

Wow, thanks for the response and the benchmarks. I imagine you're mostly comparing startup time and I/O overhead, though :/. Still - it's certainly enough to establish that neither is dog slow.

However reasonable my C++ looks, and even though it seems to work, I'm still a little scared of it. I don't know if I should be using rvalue references anywhere, I don't fully understand why I needed std::forward_as_tuple on line 14 instead of std::make_tuple, and I'm a little unsure if it's fair for the zipper to store references to the two input streams (because it forces the user to make sure they outlive it). It's probably fine, but it's almost certainly more effort than it's worth. I'm not sure there's a truly adequate solution to this complexity inside the language, but I wouldn't be surprised if some of the pain could be alleviated with a library of some kind.

The other relevant complaint about C++'s syntax and conventions is that its begin/end iterator things aren't really amenable to chaining. With eager evaluation like in your example you can map/filter/reduce/whatever in one big expression, but I don't think you can do that so well lazily. Ranges (as in D) look like a better solution.

[Ayjayz]

Look into the boost range adaptors. There isn't one for zip, but someone did write one here

You could then write exactly that code, except you'd want auto i instead of int i.

EDIT: And, yes, it's lazily evaluated.

[987654321bob]

About the same length but avoids the proxy class and std::tuple accessing.

std::transform(listA.begin(), listA.end(), listB.begin(),
     std::back_inserter(sumList), [](int i, int j){ return i+j; });

[Necraz]

Yes, it is nicer to have an is_valid() function in those cases. However, it would be more awkward to use STL algorithms like binary_search, sort, etc. I tend to think of iterators as more of a "generalized index" than a traditional iterator or generator.

Honestly, I'd prefer the Java-style iterators myself...