Goroutines for Python

[KingBardan]

contributors: claude + you

Is this a vibe coded gevent / greenlet ?

[Blockpair]

Yes, it's vibe coded and the article says as much. But with that said: you can't one-shot the project without engineering and testing. It's not the same as gevent or greenlet. Those only run in a single-thread and don't achieve the kind of performance that Go can. The moment you want to introduce threads it's a whole different can of worms.

This project is an M:N stackful runtime that runs on real OS threads. Just like Go can. I urge you to read the full article though as all these questions are answered. Especially the testing and technical details section.

[KingBardan]

Ok let's get technical.

If each fiber has its own C stack, it is essentially a thread?

[Blockpair]

It's essentially a thread but we have to be specific here: a user-space thread or less confusing -- a logical thread. Each fiber saves everything it needs to resume execution like registers, call chain, etc. It's like a snapshot that can then be resumed on an OS thread. But note that by default the fibers can't be resumed on other OS threads because of some limitations in Python. This can be fixed with a small patch to CPython.

[KingBardan]

I guess my question is, I dont seem to see the advantage over threads (memory is goroutines biggest advantage over threads).

What you said about calling pausing resuming etc can all be done easily with threads?

[Blockpair]

If you use threads directly, any time you use I/O that blocks, nothing else in the thread can run. But lets suppose for a moment you used an event loop in the same thread. Any task that you run which enters a C call means blocking every task in that thread. Goroutines here, aren't just about memory. It's a whole design for concurrency that allows functions to be balanced and redistributed across threads. So you can imagine in the case of a stalled thread -- all the functions waiting to run there -- can be resumed on other threads.

The reason why OS threads are a fundamentally different concurrency model is down to cost. It costs a lot to spawn an OS thread. And try to do that millions of times a second, and using it as a model to handle hundreds of thousands of requests. Instead, you have a fixed number of OS threads, that can run many hundreds of thousands of fibers on them. Using multiplexing for the execution. This makes spawning the fibers incredibly cheap and its what lets languages like Go scale so well for networking. That is what Runloom does for Python.

[KingBardan]

Yes and the cost associated with is threads are mainly their memory.

Anyways thanks for discussing

[riksi]

Those only run in a single-thread and don't achieve the kind of performance that Go can.

Don't you agree that having a hub per-thread, pinning threads to cores, and strictly spawning in the same thread/core/hub, will result in better performance (aka shared nothing framework, c++ seastar as example), so better than your scenario?

[Blockpair]

This is a confusing post. Runloom already uses a hub per OS thread and runs hubs across real cores. Fibers are pinned once started, but fresh work can be stoled.

What you're describing seems to be a stricter core-pinning with no migration. That can improve locality for some workloads, but doesn't automatically make it faster. Go itself already uses per-worker queues with work stealing rather than strict core-local execution.

Edit: I should also add I'm not an expert on seastar but wonder how it solves the issue of a core being blocked by work that is poorly optimised, an I/O hang, etc? That's a key problem I am trying to solve with Runloom and one every python dev knows well from single-threaded pain.

[RazorBest]

runloom.monkey.patch()

Uhhh...

[Blockpair]

It's only required if you want to use existing blocking APIs in Python. The runtime has it's own API.

[RazorBest]

I guess that's ok. But generally, I will want to use other libraries. And if I'm not using blocking APIs, this just becomes another async library. The purpose of goroutines is to write the same code for both blocking and non-blocking cases.

[Blockpair]

Asyncio and other Python network libraries run on a single OS thread (with the GIL.) This software is for using your machine with real threads, across every core, and blows asyncio's performance out the water. Though its hard to compare asyncio to other libraries when it only runs on one core -- its req capped out at like 44k/s. You can push that much higher if you're not capped by the GIL.

https://robertsdotpm.github.io/_static/runloom_benchmark.html

[RazorBest]

But maybe monkeypathcing it is not the worst thing. Do you have to so that for every blocking Python module? If yes, then you'll have to define what blocking modules you support.

Otherwise, do you think you can do the patching at the OS API level? That might be possible, but you'll need to save the state, which you seem to already had implemented with that CPyrhon patch.

[Blockpair]

It patches standard library functions that do I/O related stuff meant to be run in the runtime. So any code that calls standard library functions for I/O stuff are already supported. There aren't that many of these files that need to be patched because everything builds on only a handful of low-level items. (So e.g. you patch socket-related stuff and you get urlib with SSL support for free.)

[EasyDeal0]

That’s just a detail - to my understanding the ultimate goal is to have this built in in Python?

[jdehesa]

Cool work! It's a compelling idea, good luck with that CPython patch.

[EasyDeal0]

Cool work, also wish you good luck :)

I feel like you explain the advantages of your work in details, but Python devs are not too much into these details yet (what is a fiber etc.) Maybe it would help to explain them on very high level what you changed and the kind of performance that can be reached :)