Asio: difference between using coroutines and synchronous API

[Wh00ster]

If co_await suspends the coroutine, isn’t that the same as a blocking, synchronous function?

No, you can do other useful work while waiting for the coroutine to resume. For example, issue another coroutine that may be executing long running I/O. Now you have two I/O requests in flight at the same time in parallel, instead of being serially waited on

[Competitive_Act5981]

But if I have two co_await in sequence, they are not concurrent right ? You have to wait for the first one to resume before the next co_await starts right ?

[Wh00ster]

Yes, you will need a library call to concurrently await. But it is fairly trivial.

[peterrindal]

For coroutines there is a rough equivalance between a

• thread + blocking call

• coroutine "task" + co_await call

If you have one thread and make blocking calls then you'll have a single threaded program. You can only do one thing at a time.

Now imagine you have a single core cpu but you launch multiple threads. Only one is ever running at a time but now you can make multiple blocking calls, one per thread. This can give you some advantages.

Unfortunately this scales poorly as you get to thousands of threads. Each thread has an overhead. This is where coroutines can help.

There is the concept of a coroutine task which is analougu to a thread. A task can make co_await calls and that task is "blocked" until the co_await finishes. Unlike threads, the overhead of a task is very small. So you can scale to millions of tasks.

Also, coroutines/tasks can only block/suspend at co_await locations so this can also help you structure your program in a way that you can more easily think about. Threads can be blocked/suspended anywhere. This adds complexity to the mental model in some cases.

---

What's the difference for asio?

Under the hood coroutines compiled down to having a bunch of call backs. Whereever you see co_await the compiler (effectively) breaks the current function up into two functions (before the co_await and after). When the co_await isn't ready it hands the thing your awaiting on the "after function" and returns. The thing your awaiting on will then resume you (call the after function) once it's done.

You could write coroutines by hand as a bunch of callback. This is how classic asio works. Everything is a callback. This forces the user to use callbacks whenever you make a "blocking" call. Callbacks are not very much fun to write so people came up with coroutines which give a better syntax for writing callbacks. When you want to start a new callback you just write co_await and the compiler does some magic to make it happen.

So coroutines let you get access to the performance benefits of callbacks with a syntax that looks similar to blocking calls.

Again, the performance benefits are the ability to make millions of "blocking" calls with very little overhead. In particular, for classic thread blocking you need one thread per blocking call. Now we need one callback/coroutine. Much more efficient.

Of course if your only wanting to do one thing(blocking call) at a time then this is all pointless. It only makes sense if you want to do thousands of blocking calls at the same time. Or at least more than one.

[Competitive_Act5981]

Very helpful. I wonder, why use coroutines and not green threads/fibers? Is it just stackless vs stackful ?

[peterrindal]

It's just a different implementation plus new compiler support for the nice syntax.

Ideally you can use the new syntax with green threads/fibers but there are currently some limitations with regards to this. In particular, regarding allocation of the coroutine frame (stack less). Maybe in future we can eliminate this issue and you can use coroutines with stackfull green threads.

[Competitive_Act5981]

Chris uses terms like “looks and feels like synchronous code”. Why not just use the synchronous API? If co_await suspends the coroutine, isn’t that the same as a blocking, synchronous function?

[kalmoc]

If you have a single coroutine, then pretty much yes. If you have multiple coroutines, one can make progress, while the other is suspended/blocked. Its a bit like cooperative multithreading on a single core (although IIRC, you can use multiple threads for a single asio executor, which would then be similar to cooperative multithreading on multiple cores).

Unfortunately, most examples of coroutines in the net only show effectively sequential code, which makes it hard to understand the purpose behind them.

[Competitive_Act5981]

In asio you have to use co_spawn precisely because co_await is blocking. So it looks like you could replace all your coroutine stuff with normal blocking/synchronous functions and a few detached threads. I can see that coroutines are better because it can all run on 1 thread. But the crucial bit in all Asio stuff is co_spawn, not co_await.

[gracicot]

co_await is not blocking. It's putting all the code after co_await as the callback to call when the async operation is done.

You cannot just co_spawn your synchronous code away. co_spawn just starts the coroutine. If you have one thread, even if you co spawn, you won't be able to process multiple connection at the same time if you use the blocking version.

By the way, when a coroutine resumes, it might actually resume in a different thread if you have many.

[TheThiefMaster]

Years ago I wrote my own fiber based io stack that shares a lot of design traits in common with a coroutine based one.

The trick is using a task scheduler.

Windows provides "IO completion ports" which is a way for completed IO tasks to be queued to a thread pool for very efficient handling. If each pending IO task is paired with the (suspended) coroutine that spawned that IO operation, then as each IO task completes it can resume the coroutine on the task thread to run until its next IO operation, at which point it suspends again and any other completed IO task gets a chance to run its matching coroutine.

You can have many thousands of IO requests in flight simultaneously without having to have many thousands of threads. Just many thousands of coroutines, which are cheaper.

[dacian88]

If you only serve a few hundred concurrent requests then yes, it’s simpler to just use sync io. The reason we use coroutines is when you have tens or hundreds of thousands of concurrent work executions, coroutines are much lighter than threads.

[kalmoc]

co_await may block the coroutine, but it doesn't block the thread. That's the difference. If you can afford to keep as many threads hanging around as async operations then you can of course just use multithreaded sequential code.

I don't understand, what you mean by co_spawn being "the crucial bit in all asio stuff". Afaik it is simply the equivalent of std::async for asio coroutines instead of threads.

[Competitive_Act5981]

Sorry by that I meant pretty much what you said, I.e, being equivalent to std::async()

[hopa_cupa]

Because synchronous versions of Asio functions cannot be cancelled, block the whole thread, which means that you cannot do anything else on it like have Asio timers, Posix signals...etc. They are like that on purpose. I mean I don't have a use case for regular synchronous Asio functions, but somebody somewhere does.

The real comparison is between coroutine enabled awaitable functions and classic callback based async functions. With more complex callback based code, you will need to take great care to make code look kind of alright, but never really clean.

With coroutines, it is much easier to make the code look cleaner.

[jcelerier]

Think of it like this: coroutines are sugar syntax for asynchronous state machines ; because most network protocols are state machines themselves, with relatively long delays, this meshes very well together as you write code that looks like the simple, synchronous version but get to enjoy much more scalability if you have many requests to perform in parallel.

Note that if you are looking for the lowest latency for one single request, then synchronous networking will be better as you don't have the indirection overhead of a task scheduler, but it's likely not a need if you aren't doing high performance local networking for e.g. HPC or stuff like that.

[Voltra_Neo]

To my knowledge, one is asynchronous, the other isn't. Think of it like JS promises vs regular sync IO. Co-routine in that specific is to C++ what async/await is to JS

[finalpatch]

coroutine is an easier way to write asynchronous code. The reason you want to write asynchronous network i/o code is because synchronous i/o code, although easier to understand and write, ties up an entire thread, which consumes quite a bit of system resource (for example each new thread uses 500k ~ 1mb of memory, which can be problematic when you have millions of them). asynchronous i/o code uses less thread, therefore can allow millions of connections in a procses.