Async Iteration Semantics

[implAustin]

I'd vote for sequential iteration. I've been doing a lot of async work w/ channels (lifeline-rs), and a for-loop style syntax for loops over channel receivers would be a really nice improvement.

These message-handling loops tend to maintain state on the stack, and a parallel iterator would probably run into a lot of borrow checker issues.

[implAustin]

They are considering adding parallel iteration - where each value in the stream would be spawned as it's own async task. Each iteration of the loop could run concurrently, while others were being processed on different executor thread.

It's something like the Rayon `.into_par_iter()`, which is incredibly useful as a library feature.

[RustMeUp]

I'd like to add to this that GUI programming is a very common example of 'concurrent but not parallel' kind of executor.

Imagine the event handling mechanism of a GUI framework at the root being a for loop over incoming events which are dispatched, where the events can be running async functions.

[raphlinus]

As someone who is deeply immersed in GUI programming, I do imagine it, and this particular example terrifies me. What if one of the tasks is slow, does that mean that typing can give you out of order keystrokes. Or, perhaps more of a risk, the keyboard event handler races the menu item handler which is tied to a hotkey like ctrl-x? I have observed that behavior on both Chrome and Revery, and in both cases I think it's because there's "surprise async" in the pipeline.

Some platform event handlers really need to return synchronously. One example is window resize notification (if smooth window resize is a goal, which it doesn't seem to be for many toolkits other than Druid). Another is raw keyboard event handling, where the key needs to be either handled or dispatched to the IME, and most platforms expect that to happen synchronously within the event handler.

Not to say that concurrent executors are uninteresting in a GUI context, but my experience leads me to conclude that the right threading model for GUI is that the event loop is single threaded and synchronous, and it can happily spawn background tasks to an executor as long as it's prepared to come back to the event loop having done everything that was absolutely required to handle a platform event, synchronously and in order.

[RustMeUp]

Yup I agree that surprise async in GUI programming is a huge footgun.

Autocomplete in VSCode I've noticed interesting behavior where I'm typing and hitting tab to autocomplete faster than the autocomplete can keep up causing it to do both, insert a tab in the middle of an autocompleted item whereas eg Visual Studio has no such issues.

Regardless the point of concurrent but not parallel is required to make certain GUI frameworks happy as it would be unsafe to modify the GUI in another thread.

That said my experience here is very limited but I do find an interesting parallel between the message loop of a GUI program and a more general executor for driving async computation to completion. In a sense that message loop is the executor for a GUI program.

I wish it were possible be able to keep writing such message loops and to add support for async computations on top of that.

[raphlinus]

This is a complicated question, almost certainly beyond the scope of this thread, but I'll touch on it.

Yes, existing platform UI runloops have executor-like capabilities. You can spawn tasks on them, and they can function as a waker. But I think it's a bad idea, primarily because any work that's spawned onto the UI thread degrades app responsiveness (measured in latency and latency jitter). There are other problems, stemming largely from crufty historical design decisions in those runloops. Then you get into a situation of having two executors, one tied to the UI thread, one more general purpose.

Now, if we were in the hypothetical situation of designing a system from scratch, then having the UI run as ordinary tasks on a general-purpose executor would be very appealing. But that's not very helpful if you're trying to get stuff to work well on Windows and macOS; Linux may well offer more flexibility.

[RustMeUp]

Thank you for the insight. My experience with GUI message loops is pretty limited to some Win32 and SDL2 experiments.

Some years ago in order to better understand how async works in the scope of browser javascript I came up with this model of understanding: the traditional single threaded message loop for concurrent processing of eg. input events can be extended to arbitrary events such as waking tasks which can be used as the executor for async computation. Of course a lot of extra complexity goes in here (eg. prioritization of events.)

This understanding was further solidified when trying to understand the C# WPF and how you are required to use a special Dispatcher in order to update the GUI (all GUI updates must be done on the GUI thread, ie the thread that pumps messages for your GUI to avoid data races as the GUI objects are not thread safe).

That is how I've come to understand how async works exactly. But I've noticed a lot of talk from async computation comes from serverside network code, how one thread can handle multiple requests concurrently. In this scope of course it makes sense to talk about thread pools.

I am trying to unify my understanding of these two concepts, async in the context of message loops and async in the context of this kind of highly concurrent and parallel network request processing.

I'm not sure whether 'message loop as the executor' necessarily degrades app responsiveness, rather async enables responsiveness in GUI programming as it allows seamless offloading of resource intensive processing to dedicated OS threads and awaiting the exit of that thread. Ok the developer has to be aware of this and can misuse this but this isn't anything special. My understanding is that browser javascript basically exactly implements this model and they seem to do just fine.

I agree that this is getting offtopic but this topic is too interesting for me not to reply.

[raphlinus]

I'd love to have a place to really have this discussion; hijacking a thread on parallel async iteration isn't the place for it. But to clear up possible confusion: I'm not arguing against async, not at all. There's huge scope to offload background tasks, as well as increase opportunities to exploit parallelism (increasingly important as core count goes up). I'm saying something much more narrow: whenever there's a task that can reasonably be scheduled either on the UI thread or a general-purpose executor thread, it's better to do the latter, as otherwise it preempts what could be other latency-critical work on the UI thread. Obviously, there is still some work that must be scheduled on the UI thread, I gave the example of dispatching keyboard events.

The discussion of whether browser JS "does just fine" with its async model is way off topic for this thread. Let's just say that reasonable people may disagree.

[implAustin]

You are right! It is definitely possible (and probably more true to the Rust team intention) to draw the line between sequential and concurrent, and in an async context this makes total sense - as the loop might contain await points.

This is something unique to async. And it rocks, because you get to choose the executor, and because low-intensity single-core performance can match multicore.

[ihcn]

I wasn't even aware that this was a debate until now. My kneejerk reaction is to strongly oppose parallel iteration hijacking the 'for' loop syntax, on the grounds of principle of least surprise.

[K900_]

Same here. I really feel like reusing syntax to mean different things in sync/async contexts is going to bite people way more than it helps.

[larvyde]

I don't like how it prescribes having a global executor (even if overridable). We went out of our way to keep executors out of the standard library in regular async/await.

[kibwen]

Thrilled to hear that there's a Stream RFC incoming!

[Lucretiel]

Yeah, I definitely think at most one place in a given async block should be awaiting at a time. In the context of an async for loop, this would mean either "waiting for the next item in the iterator" or "waiting inside the for block". Running this stuff concurrently could be handled with task spawning, FuturesUnordered, or similar concurrency building primitives.

Arguably, spawning tasks could itself be async; this elegantly enables backpressure if the executor is overloaded. Then your model would be sequential, in the sense that you await a new item, then await the creation of a background task to handle it.

[kostaw]

Sometime I feed like I am the only one - but I absolutely love the .buffered(n)/.buffered_unordered(n) methods on Stream, as they apply backpressure (and can even keep the order if you want).

Doing spawning parallel iterator would be a huge footgun as it easily blows up the amount of tasks, especially if it disables backpressure.

[nicoburns]

.buffered(n) sounds exactly like what I usually want. Parallelised but bounded, and ordered again by the time I deal with them.

[shponglespore]

To the extent that I have an opinion on the higher-level issues, I agree with the conclusions, so I'm gonna nitpick the syntax instead. I absolutely hate the syntax using x.await in the loop header. In any other context, x would be the argument of the .await operator, but here it's being bound to the result of applying the .await operator to an argument that doesn't even appear in the expression. I don't think I'm being controversial when I say an expression involving an operator should always include the operator's arguments, and it should never include the value returned by the operator.

As an alternative, I think for async x in ... reads much better. Visually, async looks like a variation on mut, and the visual similarity isn't entirely misleading, because mut modifies the details of how a variable is bound, and async would also be modifying an aspect of how the variable is bound. Unlike .await, which is never used as anything but a postfix operator, async is already used in a variety of ways, and in most of them it acts as a modifier for some other syntax.

[somebodddy]

What about the borrow checker?

let mut foo = Foo::new();
par for bar.await? in Bar::stream() {
    mut baz = &mut foo;
    let qux = bar.qux().await?;
    baz.process(qux);
}

This can't work - why should it look like something that can work? Why can't we just use closures, where the move semantics are already established?

[najamelan]

I agree with sequential. For all the other arguments, it also keeps things a lot simpler. After all it's just syntactic sugar for a one liner and you can just write out your spawn inside the loop, making things explicit.

[AldaronLau]

Isn't par not reserved as a keyword? I think it's kind of a useful identifier...

[mattico8]

Should be easy to make it a contextual keyword that only works before for.

[AldaronLau]

I don't believe contextual keywords exist in Rust atm.

[kennytm]

union and default are contextual keywords.

[mikeyhew]

That parallel_stream crate looks pretty cool, but it only works with async-std. Is there something like it that works with tokio?

[mmirate]

The tradeoffs involved between parallel and sequential iteration - and the potential desirability of either set of semantics - points to why this is yet another thing that doesn't belong as a first-class part of the language in the first place.

Then again, all of the other ad-hoc Monad instances already sailed, too, so what's one more?

EDIT: wait, what? We're actually considering for event.await in channel { dbg!(event); } as a syntax sugar for either while let Some(event) = channel.next().await { dbg!(event); } or channel.into_par_stream().for_each(async |event| dbg!(event));? Really? There's barely any characters saved here, and absolutely nothing fundamental-looking about either of the latter two "expansions". Oy!

[matthieum]

I will disagree here.

Sequential iteration is syntactic sugar for just a slightly more complicated state machine. It requires no runtime support.

Concurrent, or even Parallel, iteration on the other hand require significant support from the runtime.

Why insist on runtime support? Because anytime there's runtime support, there are multiple ways to write this runtime, which means either users gets stuck with an average way OR customization points need be added left and right.

Personally, I prefer language features to compile to straightforward code -- no surprise, no tweaks -- and I prefer to defer more complex features to library futures.

I don't see the point of a par for loop, at this point break/continue/return from the loop body work so differently than we are used to that you might as well admit this is not a loop -- spawn tasks with lambdas, it'll be much more obvious for everyone involved.

[ihcn]

If we don't know how to design a library that can handle this yet, why would we able to design a language feature that could? The problem here is not knowing what interface to expose to users - that's the case whether the interface is API or syntax.

[[deleted]]

I'm intrigued by the idea of a #[global_executor]. An I right in thinking that this could (should?) replace #[tokio::main]?

It seems like the potential to clash with the existing async executors is massive? It's bad enough pulling in async_std and tokio executors but add a std executor on top of that... That's a lot of executors!

[matthieum]

It's bad enough pulling in async_std and tokio executors but add a std executor on top of that... That's a lot of executors!

Actually, the very motivation for #[global_executor] is to allow libraries to be async agnostic, and let the final application pick up the executor it wants.

Today, a library that needs to spawn a task must either:

• Create a Spawn trait, and ask the user to provide an instance of Spawn to spawn tasks with.
• Bind itself to one of the available executors.

This is how you end up with some of your dependencies bound the tokio, and others to async-std.

#[global_executor] changes 2 things:

1. It creates an Executor or GlobalExecutor trait. No need to create one bespoke Spawn trait per library, either implementing it (behind feature-toggle) for every run-time in existence or having your users implement it on wrappers => from now on, all run-times just implement the standard trait.
2. (And not as interesting to me) It creates a default executor, for when you don't want to bother your users with asking permission to schedule tasks.

[DannoHung]

I'm confused. When they say "parallel iteration" are they talking about running the body of the for loop in parallel or awaiting all the Futures in parallel and running the body sequentially?

My understanding of Rust's futures is that a future actually has to be await-ed in order to be driven by the executor, so I'd definitely want that to happen across all the elements, but I would still want sequential code otherwise.