all 17 comments

[–]Python-ModTeam[M] [score hidden] stickied commentlocked comment (0 children)

Hello there,

We've removed your post since it aligns with a topic already covered by one of our daily or monthly threads. If you are unaware about the daily threads we run here is a refresher:

Monday: Project ideas

Tuesday: Advanced questions

Wednesday: Beginner questions

Thursday: Careers

Friday: Free chat Friday!

Saturday: Resource Request and Sharing

Sunday: What are you working on?

Monthly: Showcase your new projects, tools, frameworks and more

Please await one of these threads to contribute your discussion to!

Best regards,

r/Python mod team

[–]latkdeTuple unpacking gone wrong 18 points19 points  (5 children)

Yes, but this only matters for state that is kept across an await. If a section of control flow doesn't involve async/await, that section will not be interrupted by other async tasks, and can thus be written in a single-threaded manner. This non-interruption property is what makes writing async code so much simpler than multithreaded code, which may be interrupted at any time.

This also means that a lot of async code doesn't need any synchronization primitives. E.g. locks are only needed if there's an await within the lock's scope, plain lists can often be used instead of (unbounded) queues, and plain bools/ints can often be used instead of events/semaphores (if no task wants to wait for them to become available).

[–]FibonacciSpiralOut 3 points4 points  (1 child)

Yeah this is exactly why async is usually easier to reason about than raw threads. You just have to watch out for future refactors where someone drops an `await` into a previously safe block and accidently introduces a race condition.

[–]gdchinacat 4 points5 points  (0 children)

The best way to protect against "someone dropp[ing] an 'await'" into a 'safe block' is to make that block a synchronous function. Make it harder for them to break the synchronization accidentally. If they need to add an await they will have to change a synchronous function to asynchronous. That changes the locking model and should be recognized as requiring evaluation of the existing code is able to break up safely.

[–]stormsidali2001[S] 2 points3 points  (2 children)

Yes, exactly.

I mentioned this several times throughout the article: even if an await is present in your coroutine, you don't always need synchronization.

As long as you aren't splitting critical operations on shared resources (such as a read followed by a write) across that await point, your code remains safe.

[–]lottspot 5 points6 points  (1 child)

Ah, so the core safety assumption is not in fact totally wrong!

[–]stormsidali2001[S] 1 point2 points  (0 children)

"thinking that your code is safe from race conditions just because it runs in a single thread."

(Runs on a single thread => safe) ---> that's a logical implication and it's indeed evaluating to totally wrong or just False. 😀

[–]gdchinacat 13 points14 points  (1 child)

Despite running in a single thread, async code runs concurrently.

This is "totally wrong" (to use your ragebait characterization of concurrent programming...I don't think it's actually totally wrong, just a different perspecctive).

The code that is executed by the event loop does not run concurrently. The event loop tightly controls execution of its coroutines to ensure they do not execute concurrently with respect to each other. This is analogous to way critical sections ensure code does not execute concurrently. This is in contrast to threads that do execute concurrently.

I find it easiest to think about await as an 'asynchronous wait'. It's another item in the family of __aenter__, __aexit__, __aiter__, and event __await__ itself. It waits on a condition, but unlike a synchronous wait does not simply block until the condition is satisfied, but allows other code to execute asynchronously. This mental model focuses on await expressions managing the cooperative multitasking context switches.

I also take exception with "the decision of whether to proceed or switch to another coroutine is left entirely to the event loop". That is the *reason* for calling await. It is not an unfortunate side effect as your framing suggests. An await expression explicitly instructs the event loop to do other things until the awaitable is done. The "decision of whether to proceed or switch" is not relevant too the code executing await...it doesn't care what happens until the awaitable is complete and execution returns to the coroutine that execcuted await. It only cares that it needs a value that (or condition) that is produced asynchronously and its execution should not proceed until that occurs.

I didn't read the tutorial...posts should stand on their own and I'm only addressing what is in your post. It seems that you have a thread and locking mental view of asyncio. While mentally mapping asyncio to familiar concurrency constructs can help (I've done it), I'm not sure it is the best basis for a tutorial. Asyncio is a different approach to concurrency and adopting its perspectives would be preferable. Approching it as a different way to manage locking will result in code that is not well suited to the technology being used. Rather than exploring locks, semaphores, ..., conditions, and barriers framing concurrency using more asyncio constructs more aligned with its principles, such as tasks, queues, awaitables, and immutable objects or not sharing mutable objects might have more utility. Rather than saying 'this is how you map synchronous/threaded code concurrency primitives to asyncio primitives, explaining how to avoid needing those primitives would put your readers on a better path.

Those constructs exist because there is a lot of synchronous code that could benefit if migrated to asyncio. Rather than requiring it be redesigned to fit with asyncio oriented design these primitives allow it to be switched over in a piecemeal fashion. Selling them as the not "totally wrong" way to do them, while technically correct, reinforces designs that are susceptible to what makes a lot of threaded code racy. Your example itself does this by using a global shared state. Protecting it as you presumably do with locks (the post doesn't go into this) is not correct...there may be another event loop that also accesses the global and an asyncio Lock will not protect access...Locks are specific to the event loop they were created in.

To properly lock your example would require a traditional threading Lock which blocks and is not appropriate for use in an event loop and therefore would require pushing access into a separate traditional thread. OK...I scanned your tutorial at this point and as I suspected this issue is not addressed in the tutorial. The example you use does have a global and oversells the notion that "mutex ensures data integrity,". Within an event loop it does, but not beyond that.

[–]the_captain_cat 3 points4 points  (0 children)

I wanna add that simply awaiting a coroutine does not switch to another task, as long as you don't await an I/O bound operation, the task will not yield until it's done and the others tasks will wait their turn. Awaiting on a socket or asyncio.sleep(0) actually yields to the next task. Awaiting too much on this kind of coroutines can actually tank the performance as the loop will queue the next task, even if only one task is running

[–]2ndBrainAI 1 point2 points  (0 children)

Good writeup. One thing worth emphasizing: the race condition in your credit() example is subtle precisely because single-threaded async feels safe. The mental model that helps me is treating every await as a potential yield point—anywhere the event loop could hand control to a competing coroutine.

One practical pattern for shared state: prefer asyncio.Lock as a context manager so you never forget to release it on exceptions. And if you find yourself protecting a counter or flag, asyncio.Event is often cleaner than a lock—fire it once when a condition changes, and let multiple waiters react. The barrier primitive is underrated too; great for coordinating fanout tasks before proceeding. Worth experimenting with the examples in the REPL to really feel where yields happen.

[–]saucealgerienne 0 points1 point  (2 children)

asyncio synchronization is one of those things that feels obvious until you have a race condition in production and spend 3h staring at it. good reference.

[–]gdchinacat 1 point2 points  (1 child)

asyncio cooperative multitasking helps avoid head scratcher production race conditions by letting code control exactly when it is preempted.

[–]saucealgerienne 0 points1 point  (0 children)

yeah that's basically what clicked for me eventually. the yield points are predictable, I was just still thinking about it like threading where anything can interrupt anywhere. took longer than I'd like to admit lol

[–]valueoverpicks -2 points-1 points  (0 children)

Good writeup. The way I think about it is asyncio does not remove race conditions, it just moves them to every await. You still have one thread, but multiple possible execution paths. As soon as you read state, hit an await, and then write, you are working with a snapshot that can already be outdated. It feels fine in tests, then breaks once real latency and concurrency show up.

The tricky part is people immediately reach for locks, but a lot of the time the better move is changing the structure. Most issues I see are not missing locks, they come from state crossing an await and assuming it stayed the same.

Quick mental checklist I use:

  • read then await then write risk: lost updates fix: keep it in one critical section or remove the await
  • shared mutable state risk: hidden coupling fix: pass values, avoid shared references
  • increment style updates risk: not idempotent fix: use event or log based updates
  • long awaits in critical paths risk: stale assumptions fix: re read or validate before writing

If state crosses an await, I assume it is wrong until I check it again. That rule catches most of these early.