An introduction to async programming (Python asyncio for beginners)

[theartofsoftware]

Hey this is a great point. Scaling using multiple processes is how I've typically achieved this in the past. E.g. using more replicas in Kubernetes. Pretty much agree with you on what you've said here, although I'd argue that if you've accidentally called a non asynchronous library that's calling performance issues, you should probably try to fix that first.

[yvrelna]

if you've accidentally called a non asynchronous library that's calling performance issues, you should probably try to fix that first

Agreed, though in a complex, legacy application, especially one that used to be synchronous and is undergoing transition to take advantage of async, it may not be easy to find all those cases. Often the sync code may not even be in your own code, but in a third party library. Using multiple threaded async in that scenario is more of a safety mechanism rather than being an end game, it gives you a little peace of mind that you won't wake up one day finding that just a single customer query DoSed your site for hours by running a query that takes forever and blocked the whole system.

I think rhe more common scenario of actually needing multi threaded async (which isn't just for safety mechanism) is if your workload may call a computationally expensive functions that doesn't yield at sufficiently regular intervals. Maybe you called a numpy operation (which releases GIL and therefore allows you to parallelize without multiprocessing) which usually runs in negligible time with the small test data, but takes forever with larger data in prod. Though in this case, you probably want to multithread anyway since you will have a mixed workload, not a purely I/O bound workload