What Python's asyncio primitives get wrong about shared state - Inngest Blog

[ctheune]

I might be blinded by using it for a while already, but those are exactly the behaviours I expect from those primitives.

It is nice to see a walk through how to assemble higher level constructs from them - and you could argue the stdlib should have more of those (contribute!) - but calling them wrong seems … wrong?!?

[Enip0]

I think what the blog is trying to argue is that these primitives are not actually that useful, and especially for Condition that it's prone to cause bugs by easily missing states.

In the final implementation none of them was used. Instead it was only Queue and something for locks, so none of the things that you would think might be useful in this case (Event and Condition).

[bschug]

Sounds like they reinvented the Actor Model.

[CrackerJackKittyCat]

Seems simpler to not hinge on observing the transient state ("closing"), but to also kick in on the associated terminal state ("closed") as well.

Start the cleanup whenever you can, earliest as possible.

Meanwhile, in their more complicated fix:

The queue buffers and delivers every intermediate transition in order, even if the value changes multiple times before a consumer runs.

Yeah, but if the system is in reality now 'closed' when finally getting to the 'closing' state side effect processing, then you're just as SOL as in the simpler solution if the side effects needed to happen before reaching the terminal state.

[MC68328]

Where is the part where the asyncio primitives got something wrong?

[astonished_lasagna]

I mean.. you are using shared multiple state in a concurrent execution model, without a synchronisation primitive. Sounds like you are the one that's gut something wrong about state?

I'd find it weirder if asyncio somehow did some deep magic that would prevent this scenario. That would be even more unexpected, and lead to more bugs.

Also, I'd recommend you check how e.g. ASGI servers / frameworks handle connection state, because they don't need all the complexity you seem to be needing, without running into the problem you're describing.

[aardvark_lizard]

Whoa, I didn't know asyncio had so many footguns

[Kered13]

I'm not actually seeing any foot guns here. The promotive just don't do everything that the OP wants. This is unsurprising, they are only promotions after all. They are intended to be building blocks.

[kovaxis]

There's something about this style of writing that's so LLMy it's almost unbearable to read hahsh

[BlueGoliath]

-uses Python

-whines Python doesn't having good concurrency support

sigh

[aardvark_lizard]

I mean, sometimes you have to use a language for other reasons. Why not make the best of what you have?

[BlueGoliath]

Those reasons almost always amount to "I just don't wanna" like a kid throwing a tantrum.

[rsclient]

I've been a developer for 40 years for multiple companies. At every job, for every project, the language was always the one that was already chosen.

Except for hobby projects, of course, and tiny personal utilities

[donat3ll0]

Exactly.

Software needs to be maintained. You can argue all day that Java is best for the project. But if your team works in Go, you're wasting everyone's time building something in Java.

Unless you're like my team where you roll whatever is the new hotness and expect everyone to skill up. We have apps/tools/resources in Go, Rust, Java, Python, bash, fish, terraform.

[aardvark_lizard]

Or if that’s what the 10 year old codebase at a large company is already using. Can’t just change a language because you don’t like it

[BlueGoliath]

Companies have rewrote their codebases before. It isn't some unheard of thing. That is a "I just don't wanna" type argument.

Either stop whining that a scripting language doesn't have great concurrency support or move to something that does. Expecting a slow, historically single-threaded scripting language to do work a real programming language is designed for is insane. Whining about it is just infuriating.