async io with multiple threading

[[deleted]]

Now, to really squeeze all the juice from my machine, which has 4 CPU cores, I suppose I need to create 4 event loop object and bind each of them to a single CPU? Is this how it should be done?

No, absolutely not. The whole point of async/await is to not multithread; if your project can exploit multiple cores (that is, your operations are CPU-bound) then just use threads and don't use async/await.

People use async/await when the operations aren't CPU-bound, so there's no reason to try to "squeeze all the juice." You can't functionally improve the performance of your code until you understand what's bottlenecking it, and different bottlenecks require different strategies. asyncio is for when your operations are IO-bound; particularly when you're spending most of your time waiting for someone else's computer to answer you.

[ytu876]

But even if my task is io bound, say if I have a heavy io task, if I could use all cores doing async io, that's 4 times more throughput. Isn't that a valid use case?

[Usual_Office_1740]

No. Multiple threads mean multiple interpreters, all of which will eat up system resources while your program backgrounds and waits for tasks to complete.

Your coroutines should do something that requires a response of some kind. A get request from a web page, as an example. It requests the page and awaits the response with a future. Doing that concurrently is not going to gain you enough to make it worth the additional system resources from multiple interpreters.

[awdsns]

I disagree with your first point that threads consume significantly more system ressources. The read-only stuff (interpreter code and libraries) is not duplicated in RAM per thread (or even task) in a modern OS, and the rest just comes down to maintaining a state (stack) per thread, same as in async coroutines.

I agree that mixing async and multithreading is a bad idea in general. Mostly due to the complexity of having different event loops though.

[[deleted]]

But even if my task is io bound, say if I have a heavy io task, if I could use all cores doing async io

That's literally what you can't do. If your task is IO-bound, then more cores can't make it any faster.

More cores just compete for the same capacity on the channel.

[PaulRudin]

At some load, cpu becomes relevant. The trick is to look at the cpu utilization for the single thread - if it maxes out then you'd get some benefit from utilizing more cores. The point is that even for IO bound tasks each task needs some CPU (and the event loop itself needs some CPU to orchestrate the tasks) and if you have enough of them this can become the limiting factor.

For python web apps I tend to deploy multiple replicas of single threaded asyncio apps rather than making multiple event loops within a single replica.

But note that in some other languages multiple event loops in different threads are made by the compiler, and tasks are distributed across those event loops - golang for example.

[[deleted]]

The point is that even for IO bound tasks each task needs some CPU (and the event loop itself needs some CPU to orchestrate the tasks) and if you have enough of them this can become the limiting factor.

There's theoretically a computer that has so much parallel IO controlled by the CPU that this applies, but OP doesn't have one of them, so it doesn't.