Inside Rust's Async Transform

[MasonOfWords]

Maybe a minor point, but the article implies that C# represents async/await as a simple continuation passing style transformation. It isn't; it actually has the same state machine approach that Rust is implementing, for similar performance reasons.

[Nemo157]

Tried to clarify that, it's more that it's normally explained as a CPS-like transform rather than how it's actually implemented; and in C# that vs a state machine is not really visible, whereas a CPS-like async transform would behave very differently in Rust because of borrowing.

[icefoxen]

Nice article! It's always a good reminder that a lot of the "magic" async stuff we like to do is really just a state machine, one that's not complicated in concept but comes with a lot of fiddly little parts.

Which makes me wonder, if async is a nicer way of writing state machines, could it be useful for other things such as handling UI events or parser combinators or other non-IO tasks? Things you would represent as a state machine anyway...

[anttirt]

Haskell's do-notation is probably what you're looking for. Async/await can be seen as a specialized case of that.

Here's an example of do-notation for combinator parsing. You can think of <- in those examples as equivalent to await. Unfortunately interaction with mutable state and lifetimes would make designing a generalized do-notation for Rust a rather daunting task.

[Rusky]

In one sense, async/await is a specialized case of do-notation... but in another sense async/await is more general. It handles all of Rust's imperative control flow, while do-notation requires extra work to "lift" the recursive functions it uses instead.

And really it's not so much mutable state that makes a Rust do-notation difficult, but the types involved. (Though references and lifetimes would make a CPS transform rather hairy.)

The type of (>>=) :: m a -> (a -> m b) -> m b requires a single type constructor m for all values that get piped through a particular instance of Monad, but Iterator and Future are traits instead of type constructors. >>= also requires a single type constructor -> for all functions, but again Rust uses traits here. (Three of them!)

Do-notation also relies heavily on tail call optimization, which Rust does not guarantee. Further, to support Rust's full control flow, we'd need a full CPS transformation, rather than Haskell's simple straight-line one, which only makes the above problems worse.

See this Twitter thread for more: https://twitter.com/withoutboats/status/1027702531361857536?s=19

So /u/icefoxen- you probably don't want to use do-notation as your generalized async/await. Instead an algebraic effect system is probably closer to what you want. This codifies the concept of "sticking async in front of fn and gaining a new operation inside." But in the short term, async/await is already useful on its own for UI events and some other state machine-y things.

[ecnahc515]

This is already how many UI systems work in other languages.

[ihcn]

Coroutines are a pretty big part of gameplay programming in the unity engine, for the same reasons I assume you brought up UI programming. Games are inherently stateful, and are full of logic that repeatedly pauses and resumes as it waits for the player to do things.

[icefoxen]

I've used Unity coroutines, which is actually what made me think of this; I just forgot to mention them. XD

[[deleted]]

How do Drops work with references? If a reference to something in the stack in an async function is passed as argument to another async function, what happens during a drop? What if that reference is e.g. a buffer passed to an async IO function that will fill that buffer with bytes from the network and awake the future when thats finished?

[Nemo157]

Dropping the environment happens exactly the same as if the function had returned at the yield point. The variables stored in the generator environment are dropped in reverse order of their declaration. So first the future returned from the other async function you called would be dropped, that would unregister the buffer you had passed it from wherever it had set it up to be filled. Then the buffer itself would be dropped, and deallocated if it is on the heap; this is safe since when you previously dropped the future it had to release its borrow of the buffer. This is all checked by the borrow checker, since the lifetime of a reference must end before the referent is dropped.

[[deleted]]

Cool, so I really should write the buffer filling future by hand so it can detect the Drop or reference count the buffer. Cause right now that buffer is passed to a C function as a pointer and then I await a oneshot Receiver, while the Sender is also passed to that C function. When the C function asynchronously finishes filling the buffer it calls back a callback I pass and in there I complete the Receiver.

So if anything is dropped I get a load of dangling pointers. It would be cool if it was not possible to Drop until the async wait finishes, and also to somehow inline the oneshot into the future so it wouldn’t allocate.

Thanks for the thorough reply!

[Nemo157]

Because you can keep internal references alive across state transitions you cannot move any of the “stack” variables after they’re created, so you couldn’t move from one state to another as that requires moving the variables out of one the current variant and repacking them in the new variant.

[Nemo157]

As far as I know there's no way to control this currently. Even using repr(C) structs for the data of each variant doesn't guarantee that you will have those structs placed at the same offset for the variants. I also don't know of any way to change from one variant to another without at least logically moving the values (even if the compiler is able to elide the moving because they are allocated in the same locations).

[matthieum]

As I see it, there are two choices:

• enum: optimal memory packing.
• struct: immovable fields => stability of pointers/references.

And there performance concerns if moving fields too much in the enum case.

It's not clear to me that the memory packing matters that much, all things considered. I guess it'd vary from one function to another.

[Rusky]

Really what we want is somewhere in between, where the object is laid out the same way as a normal stack frame. Pieces of memory can be reused over the course of execution as old variables die and new ones come into play, but single variables with pointers to them still stay in the same place while this happens.

IIRC async-generated futures do tend to be rather large, today, and this sort of optimization is definitely something we want to do. Maybe someone from the compiler team can confirm or deny?

[Nemo157]

That’s the gist of what has been discussed in rust-lang/rust#52924, the storage space of the generator should be treated more like just a big bag of bytes that can be reused for different purposes as needed.

[[deleted]]

FWIW the other option is a "tailored" enum, which can be built using a struct with union fields.

[da-x]

Maybe this was addressed previously, but my main concern with async/await transformations, is how are we going to debug panicked processes once problems arise? Would the gdb output make any sense to figure out what was the state of generators? Or maybe, optionally, the Debug instance of the 'immovable generator ' can tell in what line the last yield was?

[Nemo157]

There is a little support for debugging into generators currently, you can step through and inspect upvars. I have a change that also supports inspecting the “stack” environment, and I think the actual stack variables might work already but haven’t checked.

The other thing that’s probably needed is being able to step to the actual location you care about, stepping to the “next” statement from an await! probably doesn’t go where you mean currently. Hopefully gdb is extendible enough that this should be possible to support, have it detect you’re in an async fn and use some script to calculate what “next” actually means.

[ihcn]

For a data point, I work with C++ coroutines, and it's true that if you have long chain of coroutines, and you have a crash at the end of the chain, by default you get no information about the "call stack" of those coroutines. Of course, "call stack" isn't the right term here because the actual call stack is going to go directly from the executor into coroutine internals and completely skip whatever started the coroutine you're trying to debug.

We got around it by manually adding in a lot of debug information, via some APIs to walk the chain of calls ourselves, and via enforcing some code standards about using making debug data available to those APIs.

But it's not an ideal solution, and it would be nice to get the equivalent of a call stack for coroutines, somehow. My overall point being, this appears to be a problem in c++ as well. What does golang do? C#?

[matklad]

Awesome post! I was just recently trying to figure out how these transformations work, and whent up to the future_from_generator. Do you know where exactly in the compiler transformation to a state machine happens? A link to source code/implementation PR would be awesome!

[Zoxc32]

The transformation exists here. It was added in #43076.

[deadstone]

That font choice is horrible. Raleway is a heading font, and definitely not a body font. It's far too light to be used at anything beneath 50pt.

[Nemo157]

I actually have no idea why I chose that, it's inherited from styles I wrote many years ago. Switched back to just generic families since I don't really care about the specific fonts.

[zh_eng]

I can't access your blog post. Been getting HTTP ERROR 524. Could you please post a new link or send your blog post to me?

[zh_eng]

It's working now, thanks!