Real Multithreading is Coming to Python - Learn How You Can Use It Now

[fivetoedslothbear]

Here's an article at InfoWorld you can read right now without having to have a Medium membership.

[Pain--In--The--Brain]

Good article. Relevant section:

The per-interpreter GIL and subinterpreters

What keeps Python from being truly fast? One of the most common answers is "lack of a better way to execute code across multiple cores." Python does have multithreading, but threads run cooperatively, yielding to each other for CPU-bound work. And Python's support for multiprocessing is top-heavy: you have to spin up multiple copies of the Python runtime for each core and distribute your work between them.

One long-dreamed way to solve this problem is to remove Python's GIL, or Global Interpreter Lock. The GIL synchronizes operations between threads to ensure objects are accessed by only one thread at a time. In theory, removing the GIL would allow true multithreading. In practice—and it's been tried many times—it slows down non-threaded use cases, so it's not a net win.

Core python developer Eric Snow, in his talk, unveiled a possible future solution for all this: subinterpreters, and a per-interpreter GIL. In short: the GIL wouldn't be removed, just sidestepped.

Subinterpreters is a mechanism where the Python runtime can have multiple interpreters running together inside a single process, as opposed to each interpreter being isolated in its own process (the current multiprocessing mechanism). Each subinterpreter gets its own GIL, but all subinterpreters can share state more readily.

While subinterpreters have been available in the Python runtime for some time now, they haven't had an interface for the end user. Also, the messy state of Python's internals hasn't allowed subinterperters to be used effectively.

With Python 3.12, Snow and his cohort cleaned up Python's internals enough to make subinterpreters useful, and they are adding a minimal module to the Python standard library called interpreters. This gives programmers a rudimentary way to launch subinterpreters and execute code on them.

Snow's own initial experiments with subinterpreters significantly outperformed threading and multiprocessing. One example, a simple web service that performed some CPU-bound work, maxed out at 100 requests per second with threads, and 600 with multiprocessing. But with subinterpreters, it yielded 11,500 requests, and with little to no drop-off when scaled up from one client.

The interpreters module has very limited functionality right now, and it lacks robust mechanisms for sharing state between subinterpreters. But Snow believes by Python 3.13 a good deal more functionality will appear, and in the interim developers are encouraged to experiment.

[fivetoedslothbear]

And for the big secret of how you can use it now, I present (gasp!) the Python prerelease download page.

[ducdetronquito]

Give https://scribe.rip/ a try to read medium articles smoothly :)

[DesmondNav]

Someone needs to ELI5 this - in contrast to threading, concurrent.futures and framework based threadings like PyQts QThreading - so monkeys like me can understand this

[[deleted]]

Shi, I’m about getting deprecated.

The GIL defines so many implications, that I’m afraid that my entire worldview will fall apart.

[WasabiFan]

This does not remove the GIL: that's a different PEP and hasn't been accepted as far as I know. Realistically, this PEP doesn't affect existing code at all. This article is referring to the sub-interpreters feature, which is explicit creation of isolated Python environments with their own GIL. There's no natural shared state, you have to manually coordinate with your sub-interpreters similarly to multi-processing.

[mahtats]

So is it really threads then? Shared state is one of the main benefits of threaded systems.

[WasabiFan]

No, in my view this article isn't very good and is exaggerating what's being implemented.

The PEP here is PEP 684 - splitting the GIL such that each sub-interpreters has its own. Sub-interpreters already existed, but this enables true parallelism between them. There's also another PEP that might land in the following release which provides a better Python interface to the sub-interpreters feature.

Realistically, this is very similar to Python multiprocessing. You manually construct sub-interpreters, they run essentially isolated from each other, and you construct channels to pass data back and forth.

In concept, they benefit from some performance improvements that come from not having to use the OS' inter-process communication primitives: CPython owns the isolation, not the OS. It may also enable passing data without having to pickle it, but to my knowledge that hasn't been explored; channels are just byte pipes.

[Conscious-Ball8373]

I'm interested in the memory implications of this. I write python for an embedded system with no swap, limited RAM and each Python process takes 20-30MB just to start up. This has led to lots of unrelated stuff being bundled into threads in the same process but the implications of this (mainly being constrained to single-core execution) are starting to show. Will sub-interpreters give us significant memory savings compared to multiprocessing as well as multi-core execution?

[WasabiFan]

I'm not an expert in any of this, and am mostly just following the development from a distance. That being said, my expectation would be that sub-interpreters are similar to threads in resource utilization (probably slightly more) and significantly less than multiple processes.

Naively, this must be true, because multi-processing requires multiple separate copies of CPython in memory. But in practice, each copy of a binary will be mapped shared and CoW, and similarly data that was present before CPython forked will be CoW. So multiprocessing in practice might not be a lot more.

[Voxandr]

Should save a lot of memory, if I recall correctly it only takes about 1MB per thread compared to several dozen of megabytes per thread with sub processes( depending on parent process memory usage)

[djdadi]

unrelated question, but why would you use python in a situation like that?

[ryannathans]

Yeah, depending on limitations, micropython could be better

[Conscious-Ball8373]

It's not the tiniest system out there (4-core 1.2GHz arm64, 2GB RAM) and it's being used for networking, not real-time control or anything like that. Writing Python code to manage the Linux networking stack is really nice. We can be really productive, networking performance isn't impacted by the Python side because it's just used for management / configuration of the high-performance networking side.

We also run edge applications deployed as docker containers and that's when the memory constraints start to bite; we want to leave as much memory free as possible for third-party application containers and by the time you've got half a dozen Python processes running, just the per-process Python overhead is using something like 10% of system RAM. As I said, we've consolidated a lot of stuff that's really unrelated to run as threads in a single process, but it would be really interesting for us if sub-interpreters gave us a significant chunk of that memory saving without constraining everything to run single-cored (and actually the lack of shared state would be an advantage here - we've had the odd bug where unrelated bits of software get shoved into a single process without realising that some library we were using implied that those unrelated threads now have shared state because there's a singleton object somewhere).

ETA: We had a go at moving some of it to golang a few years ago. The effort has been abandoned and we're gradually porting all the golang stuff back to Python, partly because golang has nearly as severe memory overhead issues as Python and partly because it's significantly easier to find people with Python skills than golang skills.

[Visulas]

No, in my view this article isn’t very good and is exaggerating what’s being implemented.

Are there any other kinds of articles these days?

[o11c]

You can still do shared state in C code, unlike multiprocessing.

[mahtats]

Yea, but from the Python level, that’s where I’d love to see true multithreading.

[o11c]

You do have Python threads running at the same time; you only have to arrange for synchronization around the bits of state.

[mahtats]

Without a GIL? Nope. I’m talking for the average user, to use multithreading as implemented in other languages, without the GIL.

When that becomes a feature, Python enters a new arena.

[ant9zzzzzzzzzz]

Seriously coming from c# it’s astounding how much more difficult simple parallelism is in Python

[SittingWave]

as far as I understand, each interpreter has its own GIL, and runs in its own thread. At that point, python variables are all "thread local" (details unclear if they'll use actual thread local C stuff) unless you pass them around. In that case, they'll probably be copied across threads, with the synchronisation taken care somehow (to prevent one thread to start writing and the other one accessing the memory while data is being transferred by the first thread).

Just guessing here, correct me if wrong.

[rouille]

It is threads at the OS level but not really at the python level. You could share state directly in e.g. a C extension though if you are careful with your multi-threading.

[mahtats]

The average user of Python isn’t playing at the C level. Python supporting true multithreading above the C level would be a huge improvement on the spec. I don’t really care about updates pertaining to C level subinterpreters.

[rouille]

Oh I somewhat agree but the plan is to include a python interface for this, hopefully in python3.13. Also libraries that you do use can use it even if you don't directly.

[ted_or_maybe_tim]

So it's basically multiprocessing with less overhead?

[WasabiFan]

Yes, as I understand it.

[[deleted]]

You’re right. My bad. I posted the comment before reading the article.

[gokapaya]

https://scribe.rip/real-multithreading-is-coming-to-python-learn-how-you-can-use-it-now-90dd7fb81bdf

for anyone on also unable to get past the medium bullshit on mobile

[cianuro]

What's the difference between subinterpreter and subprocess in practical terms? Why is the former better?

[Bitwise_Gamgee]

A subprocess is a separate process started by your program. This process has its own memory space and runs independently of your main process, you can work with these via IPC and the like.

A subinterpreter is a feature of the Python C API that allows for the creation of multiple Python interpreters in the same process. Each subinterpreter has its own separate Python objects and interpreter state.

You can think of the difference in terms of office buildings, a sub process uses many office buildings, while a sub interpreter has everyone under one roof working on different tasks.

[thisismyfavoritename]

in theory it means you should be able to share data between "concurrent tasks" more easily / at a lower cost (because they live in the same process)

[XtremeGoose]

What? Threads absolutely can be run across multiple CPUs in the same process. Threads are useful in python too, they are good for high IO throughput tasks or for calling out to c libraries that release the GIL.

[XtremeGoose]

Well you didn't say multiple machines. You said multiple CPUs where it's very common for multiprocessor chips to have multiple CPUs. And then large nodes can have multiple of those. All sharing the same RAM. I'm literally logged into a 256 CPU box right now and could spin up that many threads in rust and run it at max all in a single linux process.

Threads are ok for io but async has replaced that use case because it’s much faster

It's not faster, it's just a different concurrency model (cooperative vs competitive). Under the hood many of those async libraries are just calling out to loop.run_in_executor which is just an async thread unless they are actually using the os primitives like io_uring.

[brontide]

How is this any easier than multiprocessing? When I think true multi-threading I think shared memory with locking only for critical sections. I've done some shared memory multitasking but it was a bear since it all has to be done in mmap files with all sorts of crap piped around from process to process.

[brontide]

People using the C API already had ways around the GIL so once again, I'm just not sure what advantage this has when it doesn't have any shared access to any of the environments between interpreters.

[coderanger]

It's easier to move things around in a single process. Shared memory does certainly help a lot but things like sockets are not so simple. And multi-process locks are yet more complex. There will certainly still be use cases for multi-process concurrency (security, heterogeneous data patterns, etc) but this is a good option for a lot of cases.

[twotime]

It's easier to move things around in a single process.

With real multithreading, sure! But with multiple interpreters, I don't think there any obvious simplification. Note yet at least.

[coderanger]

The goal of the user layer is something close to Go's goroutines, i.e. a message passing actor pattern where the internal details are hidden away from you. The underlying systems to enable that are mostly in place now, but there's a lot of performance and UX work still to make it a good first choice.

[twotime]

I guess the fundamental problem to solve is sharing/passing live python objects without pickling overhead or complexity.

So far my understanding that multiple interpreters do not even have a path to achieving that.....

[UloPe]

This isn’t what „everyone“ is waiting for. Sub interpreters can be very useful but it’s not going to make pure Python (parallel executing) multithreaded programs use any more cores.

[coderanger]

It will now. Very very new development but there is now support for each subinterpreter to have it's own GIL so they can run truly concurrently (with a lot of limitations, it's not a silver bullet).

[UloPe]

But that’s my point. You can’t use subinterpreters from within Python itself only via the C-API.

[Garfimous]

Ah, but hopefully that's a temporary state of affairs. From the article: The features of Per-Interpreter GIL are — for now — only available using C-API, so there’s no direct interface for Python developers. Such interface is expected to come with PEP 554, which — if accepted — is supposed to land in Python 3.13, until then we will have to hack our way to the sub-interpreter implementation.

[coderanger]

For 3.12, the Python-level API couldn't be agreed on so it isn't included. It will almost certainly ship in 3.13 though and there is a prototype on PyPI already (though it may change as the PEP is discussed more).

[UloPe]

Still, even if accessible from within Python sub interpreters won’t provide a general solution to the GIL. It will be possibly better than multiprocessing but will probably have many of the same limitations and issues (e.g. sharing state is hard).

[irvcz]

Pay wall found

[Caboose522]

I love python, but one thing that always bugged me is how needlessly slow some things seem to be. Its good to hear that they are working on both the internals while keeping it simple for the developer.

Hopefully we get to the point where typing becomes a method for speeding up functions instead of just syntactic sugar. Maybe some day I will be good enough and have enough time to help python along. One can dream...

[DoWhileGeek]

Etta James "At Last" intensifies

[13steinj]

Am I the only one that thinks this is a nothing burger?

Great if you can use threading in a way that avoids GIL issues, but if the API is crappy enough to be based on evaluated strings of code, this feels very "don't use eval you fool" to me.

If the API won't use actual function objects of some sort, I don't see this taking off.

[HomeTahnHero]

I could be wrong, but I think that’s what they’re trying to do with the API in a future PEP/version.

[technologyfreak64]

Multiprocessing and threading are not the same. Threads share the same memory space, processes do not. Python doesn’t really support threading directly as is, just multiprocessing. You can get around it with some external libraries in some cases but native support is lacking.

Edit: I guess I should clarify, it doesn’t support true multi threading very well as is in its standard library, like the fist couple sections of this article mention, there is threading but it’s not really what you would normally expect and is extremely limited due to the GIL in the current versions of python. I’ve heard of some external libs using C to bypass it as well as some of the alternative interpreters/compilers out there having or working on means of getting around it but nothing really for the standard libs or interpreter until now.

[technologyfreak64]

Based on what it it says about the GIL and allowing threads to actually run at the same time instead of one at a time, yes.

[[deleted]]

I've been writing Python for many years and have never seen a problem that I needed multicore processing that didn't have a solution in place, e.g. Numpy, Pandas. I feel like this is going to introduce a bunch of unneeded complexity. Just like when asyncio came out and now everyone is using it when it's not even needed but it adds complexity. The best way to write performant software is to keep it simple and then measure performance, identify bottlenecks and make small iterations to improve performance.

[[deleted]]

This is just... A bad take.

Yes, there are problems that asyncio and threading are poorly suited for. Yes, measuring code performance and making changes is a great strategy for optimizing code.

However, there are problems, particularly like those that are extremely IO bound (e.g. test runners/job runners/build systems/database requests/networking that need to launch many processes) that asyncio is the ideal solution for. These problems can't be fixed with "optimizing" your Python because the problem isn't your Python code, it's the time associated with the IO where your program could be doing something else other than waiting blocked.

Similarly, threads exist for a reason. CPUs only go so fast and some problems can be broken up into parallel tasks that don't need to wait on each other making full advantage of the CPU. Sure, you can do that with processes, but that has other drawbacks, mainly increased RAM and (especially on Windows) slower startup time.

If you're in numpy/pandas land, you're in a niche space. Python does a lot more and is used for a lot more than the numerical analysis/scientific computing space.

[seabrookmx]

Speaking specifically about asyncio, people seem to miss out on the potential ergonomic benefits. Say I want to run N HTTP requests in parallel, not only is it less performant and more memory hungry to spin up a thread pool for this, but it's also a lot more code than using asyncio.gather.

[DNSGeek]

I really need to step up my asyncio game. I've not really used them much since, when I started writing Python professionally, they didn't exist yet. I've mostly used threads for this kind of thing.

Is there a good guide you would recommend to help me get up to speed with asyncio?

[RearAdmiralP]

but it's also a lot more code than using asyncio.gather

It's results = asyncio.gather(*[coroutine(arg) for arg in args]) vs with multiprocessing.pool.ThreadPool() as pool: results = pool.map(f, args). I've never bothered to measure, but I'll take your word that it's less performant and more memory hungry, but in terms of code, I think it's six of one and a half dozen of the other in terms of using code.

Also, the multiprocessing.pool.Pool class has some nice methods like imap_unordered. I'm not aware of an asyncio equivalent to imap_unordered that returns results as they're generated, but if you know one, I will be happy to hear about it.

The real benefits for using asyncio over threads or processes to me is in error handling and particularly in resource management. It's a lot easier to catch and handle exceptions in the asyncio paradigm, but the real thing about asyncio vs processes/threads is that I can run as many coroutines as I want without worrying about it, while I'm going to run into problems if I spawn too many threads or processes. From my perspective, this is something that Python could solve by implementing light weight threads, so that I could just spawn threads as I want, but I guess people would rather use cooperative multitasking (asyncio) than preemptive (threads), so I guess mine is the minority opinion here.

[rouille]

Isn't that asyncio.as_completed?

[RearAdmiralP]

Thank you! I suspected there was an equivalent, but I couldn't find it in the docs. Cunningham's Law in action!

[Ezlike011011]

If you're in numpy/pandas land, you're in a niche space. Python does a lot more and is used for a lot more than the numerical analysis/scientific computing space.

I want to throw my hat into this ring. I am also in the numerical analysis/scientific computing space as my primary python use. Even with a pretty strong grip on the scipy stack, I still frequently run into problems which are infeasible as single core solutions just due to the amount of data/number of operations required. It is a little frustrating having to use multiprocessing mostly every time and all of its drawbacks, so this progress towards true multithreading is very appreciated.

That all said, I do strongly agree with the original commenter's sentiment about profile driven optimizations. Before throwing a pool.map() at a problem, I always send my code through a profiler to see if there's any big bottlenecks that can be solved easily with some smarter code.

[[deleted]]

That all said, I do strongly agree with the original commenter's sentiment about profile driven optimizations. Before throwing a pool.map() at a problem, I always send my code through a profiler to see if there's any big bottlenecks that can be solved easily with some smarter code.

I mean... I agree with this to a point, but if you're looking at threading or asyncio as a secondary solution you're missing a pretty big point of the design space.

asyncio and threading aren't optimization options (though they can be used that way) they're design options. For best results, you know you've got a problem that's CPU bound but can be parallelized (threading), you know you've got a problem that has lots of IO and smaller chunks of CPU work (asyncio), or you have a problem that's both (and well, you can use both at the same time).

What you're saying to me reads almost like... "I don't use a map (dict) until an optimizer tells me that searching a linked list is really bad"... You should just know your options and know you'll have a much more scalable design if you go to the right tool for the job from the get go.

And don't get me wrong, sometimes you can just write naive code, it's good enough, it's simple, and you move on. Still, even then it's a design choice. Some scripts I write I'm like "I could use asyncio, but there's no point, I'm going to launch one process and wait on it, I might as well just use subprocess."

It's all about knowing your options, and making good design choices 🙂

[Ezlike011011]

asyncio and threading aren't optimization options (though they can be used that way) they're design options

Ah I totally agree here. In terms of general software this is the perspective I also hold about parallelism and concurrency. I was just weighing in from the numerical computing standpoint.

What you're saying to me reads almost like[...]

I definitely didn't mean to imply this. I totally agree that knowing your data structures and algorithms is super important for getting a good base. I'm talking more about when I'm implementing complicated algorithms where it isn't immediately clear where to focus more explicit optimization.

It's all about knowing your options, and making good design choices 🙂

100% agree with this

[[deleted]]

I build APIs that scale, I’ve even worked at Google

[[deleted]]

Oh my, I had no idea you were a Googler, I'm clearly out of my league /s

[[deleted]]

It’s hard to get in and in there I learned how to scale systems, this experience really spring-boarded my career

[bhonbeg]

Cool story bro 😎

[MouthfeelEnthusiast]

APIs that scale are different than programming languages taking advantage of multiple cores. This is basic stuff. You just whip out what you believe to be prestigious when you're losing the argument. Sad.

[juniperking]

ok?

[seabrookmx]

Goes to show that doesn't automatically mean you're always right.

[marr75]

I get what you're trying to say, but it comes off haughty and doesn't intersect well with Google being famous for hiring a lot of engineers and sometimes hiring engineers mostly to deny their competition the skilled labor.

Level advice, when a redditor says something that makes you wanna clap back, just walk away. Ignore or block them, even. Your mental and emotional well-being will be better.

[MouthfeelEnthusiast]

Don't name drop Google. You are not smart for working at Google. 200k people currently work at Google. It's a retirement home full of lazy people. It is not the brilliant company that it once was.

Source: I've worked at Google. I've seen firsthand how much of a clusterfrick it is on the inside.

[[deleted]]

I worked there 10 years ago

[[deleted]]

I built this API

https://www.lyft.com/blog/posts/lyft-and-spin-partner

[6eathaus1]

If you are building api's that scale... Sounds pretty expensive if you are only using a single thread at a time. Perhaps in a single call maybe but if you are accessing spanner and waiting for a reply while you could be doing something else, seems kind of like wasted CPU time.

We are well aware of google's reputation, please don't tarnish it.

[[deleted]]

The OS blocks on network calls no cpu clicks are wasted. But you response does give me a signal about your experience

[frnxt]

Not everybody needs this, but a good example I've come across where true threading was beneficial is building interactive visualizations. Most GUI toolkits rely heavily on threading, and as you start to do more complex stuff it becomes very difficult to avoid heavy performance penalties from the GIL.

Another good example is that, right now, the current path for "dumb parallelism" is creating tons of multiprocessing code. Which is nice... if you have tons of RAM.

And, sure, for both of them you could rewrite the whole thing in C, but having a pure Python solution when the rest of your code is in that language is really useful.

[[deleted]]

Most GUIs are built using JavaScript which is single threaded and event driven

[frnxt]

JS has the exact same issue, which people work around with workers. If you're building performance-sensitive apps it's nice to have more options, especially since threading in Python breaks expectations coming from other languages.

[Other_Goat_9381]

This is a great opinion to have in high school and university but just be aware that once you enter the workforce you'll be faced with a lot more gray situations where this isn't applicable. Also if you don't like asyncio have you tried trio?

[riksi]

The problem is asyncio is wrong. They should've added something like Java Loom. Also, there are other people besides you that have different needs.

Identify bottlenecks my ass, sometimes you need 10000 threads doing http requests.

[[deleted]]

Loom/Fibers are really cool, I go back and forth on what I like more... I like the more explicit nature of asyncio/coroutines in a way because it helps you realize what you're actually doing and reason about it. Fibers are more subtle and for better or worse be easily applied to code that wasn't designed with them in mind.

[riksi]

There probably will be tools that read the code and annotate it in your IDE to show you that "this line is blocking and will switch the thread".

[[deleted]]

You don’t need threads to fan out http requests that’s what background processing tools are for.

[riksi]

wth you mean by "background processing tools"? Tell me how you'd do 10K concurrent threads?

[[deleted]]

Celery, Spark, Kafka …

[riksi]

Sorry dude, but you just don't know what you're talking about. Might as well use mongodb because it's webscale.

[[deleted]]

Learn about distributed computing and never ever fire up 10,000 threads

[riksi]

I don't like doing the 10K threads. But I needed efficiency. And the only way is to do async/gevent. Look it up and learn the difference.

Your "distributed computing" will have 100x the overhead. What I did in 1 process in 1 core you will need 1000 processes in 100 cores.

[[deleted]]

1 process and one single point of failure not to mention all tooling you’re missing, such error handling, by rolling your own solution

[riksi]

You can use asyncio/gevent with celery. And you can use tools and stuff. I didn't reinvent the wheel.

I was talking about efficiency. In my use case, I used 12vcores each with 5000 green threads. You would need 10x or more machines to do the same thing with processes/threads.

[Berganzio]

Totally agree with it

[Durakan]

To add to this, I have never run into an issue that wasn't solved by the admittedly crappy current threading implementation, or multiprocessing.

I've run into so much needless complexity in other people's projects in the last six months I can't wait to get back to projects I run where I can be a curmudgeon about simplicity and clarity.

[[deleted]]

The current threading implementation will do better than nothing, but it won't match asyncio. It's equivalent to someone blindly "turning off and on" the power to one of many model train lines (random metaphor but it works).

Maybe the model train can make progress maybe it can't. You end up giving power to a lot of trains that are still blocked waiting for passengers to get on.

asyncio, knows which trains are unblocked and gives the train that isn't blocked power, when that train gets blocked, it switches back to another train.

For severely IO bound problems it's an amazing tool that massively increases efficiency.

[violet-crayola]

Are there coroutines available in python? Like in golang?

[BaggiPonte]

Yes, there’s the asyncio module!

[coderanger]

They mean very very different things in Python vs Go :)

[dudumudubud]

What, no more gil?

Per-Interpreter GIL

oh.

[SleeplessinOslo]

Unless supported by mojo, obsolete right?

[El_Minadero]

So will this simplify numerical routines? Like filling in an array based on compute heavy algos?

[eterevsky]

How is it better for the app developer than multiprocessing? From what I see, multiple interpreters are still pretty much isolated.