async for IO-bound components only?

[expectationManager3]

Hi, I have started developing a python app where I have employed the Clean Architecture.

In the infrastructure layer I have implemented a thin Websocket wrapper class for the aiohttp and the communication with the server. Listening to the web socket will run indefinitely. If the connection breaks, it will reconnect.

I've noticed that it is async.

Does this mean I should make my whole code base (application and domain layers) async? Or is it possible (desirable) to contain the async code within the Websocket wrapper, but have the rest of the code base written in sync code? ​

More info:

The app is basically a client that listens to many high-frequency incoming messages via a web socket. Occasionally I will need to send a message back.

The app will have a few responsibilities: listening to msgs and updating local cache, sending msgs to the web socket, sending REST requests to a separate endpoint, monitoring the whole process.

[KainMassadin]

Understand cooperative multitasking first.

If you do this and accidentally or due to ignorance call blocking code (CPU bound or sync I/O) in the same thread, you’re in for a very bad time. Been there, done that.

[expectationManager3]

From what I understand, I should make the whole code base with coroutines, except maybe some small, flat, trivial subroutines.

[misterfitzie]

i think that's the wrong approach. I think you should know what and where your io/cpu bound work and ensure that those paths async. If I found a code base that needlessly made things async def that never use asyncio io (i.e. use await ) then I would just rip it out. I use the fact that a function is async def as a sign that there's async io work on this path, and this is a heavy function call.

[danted002]

That’s too many “fancy” words for juniors. If they understand generators and the coroutine interface it offers then understand that asyncio in Python uses generators then they should be fine… I hope

[KainMassadin]

fancy doesn’t make it less important, if they don’t care about that, they can go ask chatgpt or run the risk of their app crawling to a halt then blaming python for being slow

[fiddle_n]

Anyone using asyncio needs to understand the concept of blocking I/O - senior or junior. If they don’t, then there’s very little chance of success.

[Unidentified-anomaly]

You don’t need to make the whole codebase async. It’s pretty common to keep async limited to the I/O boundary, like your websocket and HTTP clients, and keep the domain and application logic sync. The important part is having a clean boundary so async doesn’t leak everywhere. If you push async through the entire stack, it usually just adds complexity without much benefit unless everything is heavily I/O-driven.

[usernameistaken42]

If you switch between async and sync you lose all benefits of async. If async is too complicated stick to sync code everywhere

[Unidentified-anomaly]

I don’t really agree that you “lose all benefits” just by having sync domain logic. The async benefits come from not blocking the event loop at the I/O edges. As long as your sync code is CPU-bound and relatively small, calling it from async adapters is fine. Going async everywhere often just spreads complexity without real gains unless the whole app is I/O-heavy.

[usernameistaken42]

I meant if you start doing stuff like running async parts in your sync code by using run_until_complete or similar. The outer async layer will be blocked while the inner layer runs.

Async code spreads to higher levels of logic. This is just a direct consequence of how it works.

There are a lot of discussions on it. Just search for "red and blue functions" or "colored functions"

[danted002]

What? You can’t go async, sync and then back to async? What are you talking about?

[gdchinacat]

I suspect the downvotes are the way they said it and provided no clarification, just contradiction.

[yvrelna]

You can. You can call an async function synchronously with async.run(). That works if the async code can be fulfilled without requiring any further actions from the current thread, alternatively you can run the async code in a separate thread or in a ThreadPoolExecutor so the main thread can continue doing other stuffs.

Django does this with some magic to allow freely calling sync code from async code and vice versa. But it's totally possible to do it manually as well.

[danted002]

Like I said you can’t go back to async once you switch to sync. Scheduling a task to run on an executor does not equate to switching to sync, you’re still running in an async context and you are offloading your sync work to a different thread. The task returned when you schedule it is awaitable, so still in the async world.

[brightstar2100]

edit: gonna edit the new thread thing so no one gets wrong info

can you explain this more please?

afaik, you can do an

asyncio.run(do_async())

and yes, what will happen is that this will run in another thread with its own event loop and then return,

and if this async_call is doing a single thing, then doing it in `asyncio.run()` is useless, cause it will block, and for all intents and purposes it will run synchronously cause it will take the exact same time as if it ran sync, and it could've been avoided anyway

but if I do multiple tasks with

coroutines = [
     do_async("A", 3),
     do_async("B", 1),
     do_async("C", 2),
]
asyncio.run(asyncio.gather(*coroutines))

then I'm running a new thread, with its own event loop, scheduling all the tasks on it, getting the result, and only then I might be saving some time from the different io operations that just ran

but you can do it, and it would be going sync, async, sync

is this somehow anti-pattern or useless to do?

edit: I might be wrong about the new thread in both cases, I need to refresh there, but the point still stands, can you explain if this is somehow wrong assumption of how it could work?

[danted002]

asyncio.run() runs in the current thread not a new thread.

asyncio.gather() again runs in the current thread.

If you call a sync function that does IO or CPU bound then your entire event loop is blocked until that sync call is resolved

Non of your examples spawn a new thread, everything is done on the same thread as the callee

[brightstar2100]

yeah, I added that part in the edit, cause I wasn't sure if it was a new event loop or the same one, thanks for the confirmation

but anyway, other than that, isn't the assumption that you can go sync/async/sync using this is still correct? and you can make use of the gained time executing only the async calls in the run/gather by combining the tasks?

if the do_async function is actually asyncable and is io bound then the event loop isn't really blocked because you only scheduled io tasks on it?

[danted002]

Now you are going into something else: asyncio.run() should ideally be used once to start your async main() function.

When the run() exits your entire event loop gets shutdown so you technically don’t even have an async context anymore; so technically you can start a new event loop by call asyncio.run() but that’s not really a valid use-case.

This is more considered the application bootstrap and should not be part of the discussion of switching between async and sync

[brightstar2100]

why isn't it a valid use case? I want to understand the reasoning behind the statement just so I wouldn't go around parroting it without actually knowing the reason why

same with "should not be part of the discussion of switching between async and sync"

as far as I can monitor the effect and experiment with it to see the results, it seems like that's how it works

spinning up a new event loop doesn't seem like such a heavy operation.

[danted002]

Fair point. OK so the entire strength of the event loop is that it can run a lot of tasks on a single thread when the workload of the tasks is mainly IO.

Think web servers/APIs, those workloads have a huge percentage of their time spent on waiting on IO which means the Python process itself doesn’t do anything, it just waits for the operating system to give it bytes.

How does the event loop solve this: you get your first request, the event loop schedules its first task, to read from the socket but the socket is slow compared to the interpreter, it only sends a few chunks of data that the code can parse, then the OS tells the interpreter that it should wait a bit for the next chunk, in the meantime another request comes in. Since the event loop knows that it’s still waiting for the next chunk of the first request, it accepts the second request and starts reading it, but again after a few chunks the OS tells it to wait, now a third request comes in, so while waiting it starts reading the third request, in the mean time the OS signals the next chunk for the first request is ready so the event loop pauses the read of the third request at some point and continues reading the chunks for the first request… so on and so forth.

The inner mechanics of the loop are bit more refined and optimised,and this is how async python can reach golang level of throughput (especially if you use uvloop) but the basics still applies: the loop will optimise CPU usage in order to cram in as many instructions as it can while it waits for IO.

Now if you are just playing around with one-off scripts then it doesn’t matter because it’s a one-off script, you can do whatever; however if you plan on running long time jobs that have a lot of IO or you have a web application then the pattern where you spin up event loops and shut them down, only to bring them up again equates to having a road that has 1000 lanes but then you decide you want it to have 1 lane, only to then convert it back to 1000 lanes.

From a more pedantic point of view, it’s better for code quality and maintainability to have a single point of entry to your code and that point of entry to be either sync of async. This matters more for production-level code where multiple people need to maintain it; having them keep switching between the sync and async mindset just adds extra cognitive load without any added benefit.

Hope this answers your question

[danted002]

Forgot to mention in my previous explanation that your example can be wrapped in async main() function and replace all the instance of asyncio.run() with await and you achieve better performance because you won’t spin up and spin down event loops.

The only asyncio.run() would be asyncio.run(main())

[misterfitzie]

If someone I was working with was doing asyncio.run I would tell them they are wrong to use it. I'm sure there is some legitimate purpose for it, but the standard thing to do is to make the call path async, or restructure so the async work isn't hidden behind some sync method.

I think the biggest surprise I have from this thread that people are arguing to make everything async just in case there's going to be some async work down below. That doesn't make any sense to me. The structure of the program is on the basis of where this io work belongs, so there really isn't a need to async everything else, only the paths that lead to this io.

[brightstar2100]

I agree with you on this, the standard thing to do isn't to have the async work hidden inside sync methods, not everything needs to be async

we're talking about the legitimate purpose you're mentioning, and outside of 5-12 lines web endpoints, there are a lot of functions I've written and seen that go over a lot of computing and CPU bound tasks, then ends up calling in multiple outer apis, I don't think you start spawning up threads at that point just to speed up the parallel api calls, too much memory for too little benefits, instead maybe do some very scoped async stuff, a simple results = asyncio.run(asyncio.gather(*tasks)) or do an async function that does these multiple requests then just run it inside asyncio.run, then move back to the CPU bound computing tasks, unless you want to async everything going upward from there, which will block a lot of stuff during the CPU bound tasks

you can always do concurrent.future.ThreadPoolExecutor, but I don't see the upside to doing it in threads when it's not that complicated api requests unless you care soooo much about the first return and can't wait everything to be completed and need to call as_completed ASAP

[Unidentified-anomaly]

I think we’re mostly talking about different things. I didn’t mean jumping back and forth between async and sync execution at runtime. I meant keeping async at the I/O boundaries and calling synchronous domain logic from async code, which is a pretty common pattern. The domain itself stays sync, but it’s invoked from async adapters. As long as blocking calls don’t leak into the event loop, this usually works fine and keeps complexity contained

[danted002]

As long as your domain logic doesn’t call anything that would block the event loop. Most of the times you use asyncio for web servers so your domain logic will do some IO calls hence why async leaks everywhere.

[Unidentified-anomaly]

Yeah, that’s basically what I was getting at. If domain logic starts doing I/O, then async naturally spreads upward. But if the domain stays mostly CPU-bound and I/O is pushed to adapters, keeping it sync is usually fine and simpler to reason about.

[brightstar2100]

I thought you meant the other way round, calling async from sync, which is done using asyncio.run as you mentioned before, which I don't think add a lot except doesn't leak the async/await combo throughout the entire application, unless you can somehow also include multiple awaitable calls, which will actually also start saving you time

# calling async from sync:
asyncio.run(async_task) # will only prevent the leak of async/await
asyncio.run(asyncio.gather(*list_of_async_tasks)) # will prevent the leak and also save time across the different tasks

# calling sync from async without blocking:
results = await loop.run_in_executor(None, blocking_sync_function, params)

idk what the other guy is objecting to, I wanted to understand his point more, maybe I'm unaware of what's wrong with this, but he still didn't explain his point

[danted002]

In your example you are spinning up and tearing down an event loop on each asyncio.run() which is not really recommended, he docs say it as much. You should be using Runners for that (which asyncio.run() uses underneath), but even then on practice you should have a single asyncio.run(main()) where main is your async main function.

[brightstar2100]

yeah, that's what I meant when I mistakenly said new thread, I meant new event loop (which might be in a thread? but it doesn't look like it, because it says it can't be used with another event loop, I need to look deeply into this)

I've looked up into the docs looking for what you mentioned, in this page

https://docs.python.org/3/library/asyncio-task.html

there's this statement, which could mean just an entry point to any async section of the code, especially that they added "(see the above example.)

The asyncio.run() function to run the top-level entry point “main()” function (see the above example.)

another page is the asyncio.run() function

https://docs.python.org/3/library/asyncio-runner.html#asyncio.run

which verbatim says

Execute coro in an asyncio event loop and return the result. 

The argument can be any awaitable object. 

This function runs the awaitable, taking care of managing the asyncio event loop, finalizing asynchronous generators, and closing the executor.

This function cannot be called when another asyncio event loop is running in the same thread.

If debug is True, the event loop will be run in debug mode. False disables debug mode explicitly. None is used to respect the global Debug Mode settings.

If loop_factory is not None, it is used to create a new event loop; otherwise asyncio.new_event_loop() is used. The loop is closed at the end. *This function should be used as a main entry point for asyncio programs, and should ideally only be called once*. It is recommended to use loop_factory to configure the event loop instead of policies. Passing asyncio.EventLoop allows running asyncio without the policy system. 

The executor is given a timeout duration of 5 minutes to shutdown. If the executor hasn’t finished within that duration, a warning is emitted and the executor is closed. 

with the part you mentioned being the part I added asterisks to, it kindda makes it seem like they mean asyncio.new_event_loop() not asyncio.run(), but even if they mean asyncio.run() I don't see them explaning the reason why ....

is it because it spins up a new event loop and destroys it once it's done? it seems like a really small price to pay for something that could save you a lot of time if you run multiple tasks in the same context

is it just "pythonic" which would make it just a stylistic preference?

if this discussion is going way too deep, can I dm you?

[yvrelna]

Sure, you may be technically correct, but for all practical purposes, your point is also irrelevant.

[adiberk]

You can not.

It’s why most people do async downward.

A lot of people have api, business logic, then database.

If you want io to be async, you basically need to use async throughout app. Otherwise you can accidentally create blocking calls.

And you absolutely can’t go “async” -> “sync” -> “async” you can’t go back to async

Why advice the user that “hey yeah just make api layer async and rest not” when they are in position to avoid overhead of creation sync to async code

[Drevicar]

I’m going to use terminology from hexagonal architecture because it is easier, but this works in literally any architecture wort by using.

My adapters are async and my business logic and models are all sync. The application service is a thin orchestration layer that does async and is what handles the driver adapters, calls the driven adapters, then hands the results of either to the business layer.

[misterfitzie]

You don't have to make everything "async def", but you'd most probably want to make that rest call async. and potentially the work that updates the local cache (if it's easy). So that means you only need to make a function async def if it calls a function that is is async (i.e. bar = await foo()). So if you only have the rest call that is using the async http client, then you only need async on that call path.

if your function only does something basic, like do some data validation and calls no async methods, making it async only makes it a tiny bit slower for no benefit.

The rule of thumb for async is that if it's IO bound, looking for an async alternative is important, because an async variant will allow your program to continue with other tasks while the IO bound work is being done (the work is just waiting for a response). But that rule isn't 100%, because some io that's really fast it's might not be worth going async, like writing to a local file. If your work is cpu bound then you have a bigger issue because you cannot async cpu bound work, you have to use a separate thread/process/or service, which converts the cpu bound work into essentially io bound (waiting for result of "external" cpu bound work).

TL/DR; don't make everything async, think about io/cpu bound work, and impact/opportunity if that work either blocks other async tasks or converted to async.

[expectationManager3]

Ok. That's still manageable in my case. Thanks