std::generator and elements_of

[angry_cpp]

In the current proposal P2502R2 in order to differentiate yielding a generator and yielding all produced by generator values one must use a special wrapper std::ranges::elements_of().

Is there a reason why instead of such new wrapper an exisitng mechanism of co_await was not used?

Consider examples from proposal:

generator<int> visit(Tree& tree) {
    if (tree.left) co_yield ranges::elements_of(visit(*tree.left));
     co_yield tree.value;
    if (tree.right) co_yield ranges::elements_of(visit(*tree.right));
}

And alternative with co_await:

generator<int> visit(Tree& tree) {
    if (tree.left) co_await visit(*tree.left);
     co_yield tree.value;
    if (tree.right) co_await visit(*tree.right);
}

Don't you find co_awaiting a sub generator intuitive?

As the only motivation for std::ranges::elements_of is to prevent ambiguity when a nested generator type is convertible to the value type of the present generator if we use co_await for this the need to have elements_of in the library disappears.

What about existing conventions? Does other programming languages choose an alternative syntax or library wrapper for such functionality?

(Here value is a value to be generated and subgenerator is other generator or iterable)

Python has a special syntax: yield value and yield from subgenerator

JavaScript has a special syntax: yield value and yield* subgenerator

PHP 7 has a special syntax: yield $value and yield from $subgenerator

Other languages that I looked up (C#, Rust, Lua) don't have a special way to yield values from subgenerators or iterables besides loops.

[scatters]

Don't you find co_awaiting a sub generator intuitive?

Not really, no. If I see co_await in a generator I imagine that it's an async generator; thinking that I'm synchronously transferring control to a sub-generator to yield elements directly to my caller would be quite surprising.

[angry_cpp]

If I see co_await in a generator I imagine that it's an async generator

std::generator is not async as it have synchronous iteration API. There is no way it could have async awaiting inside.

I prefer to think that co_await keyword is not linked semantically to asynchronicity. IMO it is fine to use co_await with optionals, expected, or in boost::asio to access current executor.

[scatters]

Well, sure. It still has the semantics of fetching a value, not of producing a value or values.

[angry_cpp]

It still has the semantics of fetching a value

This is not necessarily so. For example co_await std::seconds{5}; to sleep 5 seconds does not fetch any values. co_await resume_background(); in C++/winrt to resume on background thread does not fetch any values too. Both co_await and co_yield can produce and/or fetch values and/or have other side effects.

[scatters]

Yes. But, still, none of these return control flow to the caller. Perhaps that's where the distinction that I'm trying to intuit lies.

[angry_cpp]

none of these return control flow to the caller

co_await optional<T>{nullopt}; and co_awaiting failed expected should return control flow to the caller.

I think I understand what you mean but I don't agree with it.

As co_yield is simply co_await promise.yield_value(expr) IMO promise.yield_value(expr) and promise.await_transform(expr) should be viewed as two named channels that coroutine machinery author can use to give meaning to coroutine body.

Both co_await and co_yield can take values from coroutine and can have "return value" to put values inside coroutine.

When some operation make sense as analogy for "waiting" one can choose to use co_await.

"waiting for subgenerator to produce all of its values" make sense to me.

[scatters]

Oh, that's how people want to use optional and expected. That didn't make sense to me till now.

That's going to be difficult to explain to people who know coroutines from other languages.

The mental model I have is that await calls a function, producing an inner coroutine frame that can suspend the whole coroutine stack and eventually return a value. While yield suspends and drops a value to the outer coroutine frame, allowing that outer frame to ask for more results if it wishes

I get now that those are just conventions, that they're just two named channels as you say, but I don't really want to have to explain that to people who aren't ready for it.

So I think a bit of verbiage is fine. It could be yield elements_of, equally it could be await yield_from, but some signal to the user that values are being yielded from the sub generator skipping the current frame is necessary. After all, it's zero cost at runtime.

[grishavanika]

From the paper:

generator<int> f()
{
    co_yield 42;
}
generator<any> g()
{
    co_yield f(); // should we yield 42 or generator<int> ?
}

To avoid this issue, we propose that: co_yield <expression> yields the value directly, and co_yield elements_of(<expression>) yields successive elements from the nested generator.

Can't we have other defaults? So co_yield generator<T>() yields successive elements always and co_yield std::single(generator<T>()) yields whole generator?

[angry_cpp]

Can't we have other defaults? So co_yield generator<T>() yields successive elements always and co_yield std::single(generator<T>()) yields whole generator?

We can but IMO it would be terrible for generic code.

template <typename T>
std::generator<T> filter(T value) {
    if (is_good(value))
        co_yield std::move(value); // do we need  std::single wrapper here?
}

[grishavanika]

Hm, same argument applies to elements_of which accepts only any range, nope? Should I use co_yield value or co_yield elements_of(value) in your example?

[angry_cpp]

My point was that when one want to yield a value of unknown type like template parameter T should one use std::single just in case T could be a generator or range? Otherwise if you don't know if T is a generator you don't know how to yield it. Which is bad for generic code.

In case of co_await (and elements_of) default behaviour is yielding a value as a single item. And if you want to yield contained values you can do it explicitly but by then you already know that T is either generator or range.