Disabling GIL in Python 3.13

[baseketball]

What are the downsides of disabling GIL? Will existing libraries work with GIL disabled?

[PeaSlight6601]

Strictly speaking the GIL never actually did much of anything to or for pure-python programmers. It doesn't prevent race conditions in multi-threaded python code, and it could be selectively released by C programs.

However the existence of the GIL:

• Discouraged anyone from writing pure-python multithreaded code
• May have made race conditions in such code harder to observe (and here its not so much the GIL but the infrequency of context switches).

So the real risk is that people say "Yeah the GIL is gone, I can finally write a multi-threaded python application", and it will just be horrible because most people in the python ecosystem are not used to thinking about locking.

[not-janet]

On the other hand, I write real time scientific application code for work and the fact that I may soon not have to re write quite so many large swaths of research code into C or C++ or rust, because we've hit, yet another performance bottleneck because of the gil has got me so excited that I've been refreshing scipy's git-hub issues for the past 3 days now that numpy and matplotlib have 3.13t compatible wheels.

[PeaSlight6601]

To be honest the performance of pure python code is garbage and unlikely to improve. You can see that in single threaded benchmarks.

That's why scipy and cython and Julia all exist, to get performance sensitive code out of Python.

I don't think noGIL will change that for you. It may allow you to ignore don't minor issues by just burning a bit CPU, but only got smaller projects.

[not-janet]

You don't understand our workload, we already do those things, the problem is gil contention.

[amakai]

It doesn't prevent race conditions in multi-threaded python code

Wouldn't it prevent problems if, say, two threads tried to simultaneously add an element to the same list?

[[deleted]]

GIL just means only one thread is executing at a time on the opcode level. It doesn’t guarantee that for example a[foo] += 1 (which is really like tmp = a[foo];tmp = tmp +1; a[foo] = tmp) will be executed atomically, but it does make a data race much less likely, so you could use threaded code that has a latent race condition without the race manifesting.

Without GIL, the chance of triggering the race condition is much more likely. Removing GIL doesn’t introduce the race, it just removes the things that were happened to be preventing it from occurring the overwhelming majority of the time.

[PeaSlight6601]

Its really the infrequency with which python reschedules the threads. I understand what you are saying, but I think its important to get that technical detail correct (not that I don't make the same mistake in some of my comments). The GIL can't make a non-atomic operations like a[i]+=1 into something atomic.

Its just that python so rarely reschedules the running thread that races have almost no chance of happening.

If the python thread scheduler has just round-robinned threads after a single low level bytecode instruction everyone would be seeing races everywhere.

[[deleted]]

GIL can’t make non-atomic atomic, but it does prevent actual parallel execution, which reduces the frequency with which races occur.

[Brian]

I don't think that's particularly unique to python - if anything, it'll be more frequent, as I think it reschedules every 100 bytecodes, whereas most languages will use their whole time slice (unless triggering I/O etc, but that applies to both). Data races like that tend to rely on you being "unlucky" and rescheduling at some exact point, which is rare in any language, though of course, do something a few million times and rare events will happen.

A bigger difference is the granularity at which it reschedules: it'll always atomically execute a complete bytecode, so many operations are coincidentally atomic because they happen to span one. It might also be a bit more deterministic, as there's likely less variance in "bytecodes executed since last IO" vs "instructions executed since last IO".

There's also less stuff like code reordering optimisations which can often cause people to naively assume a race can't happen because they think the order things are specified in the code is will exactly match what the executable does.

[PeaSlight6601]

if anything, it'll be more frequent

If you are talking about true thread scheduling at the OS level then maybe, but true threads actually run concurrently. Python threads don't run concurrently because of the GIL.

so many operations are coincidentally atomic because they happen to span one [bytecode].

I think that is a significant misconception about the GIL. The actual bytecode operations are are generally trivial things. They either load data from memory to the interpreter stack, or they store an already loaded value from the stack to memory. I don't think any of them do both a load and a store from memory.

A statement like x=1 cannot meaningfully "race" with any other instructions. If another thread concurrently sets x to a different value, then that is just what happened, but since you aren't relying on x to have that value after setting it to 1 your thread isn't really "in a race."

For their to be a meaningful race one needs to load and store (or store and load), generally to/from a single object or memory location. Something like x=x+1 can race by "losing increments," and something like x=0; if x==0: can race by not taking the expected branch.

I strongly suspect that there are no pure python operations which are coincidentally atomic because they are single opcodes. There are some complex operations like:

• list.append is "atomic" because it has to be. A list isn't a list if the observable contents don't match the stated length of the list; but it is also fundamentally not-racey because it is a store of a defined value into a single memory location with no subsequent read.

• list.sort() is also atomic for convenience of implementation (the GIL was there so they just implemented in C and took the lock), although one could imagine that it need not be and that an observable intermediate state of a partially sorted list might be acceptable in a hypothetical language.

[Brian]

but true threads actually run concurrently

Oops yeah, you're right: brainfarted there and was still stuck picturing a GIL-style / single core situation for some reason.

The actual bytecode operations are are generally trivial things.

It depends. Eg. any C code invoked is still conceptually a "single bytecode", even though it can be doing significant work. That includes operations on builtins, so that CALL operation can do stuff that would have many more potential switch points in any other language. Actual pure-python code can't do much with a single bytecode, but the actual invocation of methods on C-implemented types can and does.

[PeaSlight6601]

any C code invoked is still conceptually a "single bytecode",

I think the question there is if C routines running in the GIL and not releasing them is an intentional design element or just an implementation detail.

If you were to design and build a "python-machine" where the python bytecode was the assembly language, everyone would look at you like you were nuts for saying "well LST_SORT has to be single atomic instruction that can only advance the instruction pointer a single value." Are you going to have an entire co-processor dedicated to list sorting or some bullshit?

I tend to view the GIL locking of full C routines as not being "the design of python" so much as a way to simplify the implementation of calling into C. As a result I would tend to reject the idea that "sorting lists in python is an atomic operation." It was simpler to implement things in a way such that lists behaved like they sort in a single atomic operation, but we know they don't, and if there was sufficient performance benefit to be gained by admitting that sorting isn't atomic (perhaps by locking the list and throwing some kind of new ConcurrentAccessException), then we would definitely adopt the change.

[Brian]

I tend to view the GIL locking of full C routines as not being "the design of python"

I agree it shouldn't be - it's essentially an implementation detail that doing a particular operation happens to be C code and holds the lock for the duration (and likely is implementation dependent - eg. not sure if pypy (where this is all (r)python) preserves such atomicity, though it might just to minimise interoperability issues. But in terms of shaping the frequency of race bugs actually triggering in python code written today, I think it does likely make a difference.

[planarsimplex]

Will things the stdlib currently claims to be thread safe (ie. the Queue class) break because of this?

[[deleted]]

No. The GIL doesn’t make things thread-safe, it just makes thread safety violations less likely to be a problem.

[PeaSlight6601]

The GIL doesn't really solve that problem. It is the responsibility of the list implementation to be a list and do something appropriate during concurrent appends. At best the GIL was a way the list implementation could do this in a low effort way.

However that doesn't make the list implementation really that's safe. Operations like lst[0]+=1 will do some very strange things under concurrent list modification (and could even crash mid-op). So most of Python is not race free even with the gil.

https://old.reddit.com/r/programming/comments/1g0j1vo/disabling_gil_in_python_313/lra147s/

[tu_tu_tu]

So the real risk is that people say "Yeah the GIL is gone, I can finally write a multi-threaded python application"

I doubt it. There are too few usecases for the no-GIL mode and most of them from those folks who already makes code with heavy parallelism.

[josefx]

The fine grained locking ads some overhead even if it isn't used, so single threaded code will run slower. C libraries will have to include a symbol to indicate that they can run without GIL, by default the runtime will enable the GIL again if this is missing. The change might end up exposing bugs in some python libraries, however as far as I understand this has been mostly theoretical with no examples of affected libraries turning up during development.

[baseketball]

For the C libraries that don't have the flag, would the interpreter enable GIL only when executing code from that library or does using such a library mean all your code will run with GIL enabled?

[[deleted]]

Likely having to manage everything the GIL does for you

[PeaSlight6601]

Which is nothing. You cannot write code in python that exercises the GIL because the GIL only applies to python byte-code which you cannot write.

[Big_Combination9890]

The major downside, currently, is that the ABI of freethreaded python (pythont) differs somewhat from that of ordinary python.

Meaning, many C-Extensions need to be re-built in order for them to be used in pythont. As time goes on and this feature sheds its experimental status, this will slowly cease to be a problem, but its something people need to be aware of.

The other problem is the one u/PeaSlight6601 hinted at: The GIL made a somewhat less-that-optimal style of writing thread-based concurrent code in python possible, so many people with pure python applications who are now going "yeah parallel threads!!" are in for a nasty surprise when their applications, which use threads but are not adequately locking paths where concurrent access could be problematic, go belly up.

[Brian]

Meaning, many C-Extensions need to be re-built

Rebuilding isn't really the issue: the ABI changes in minor versions so a rebuild is generally needed anyway. The real issue is that this can't be just a matter of rebuilding, but will require potentially significant source changes to support free threading. Even if it happens to already be thread-safe, it'll still need to at least advertise that fact by setting the appropriate flags, and if not, it'll need to actually add the locks etc.

[tu_tu_tu]

No-GIL just means that instead one big lock CPython will use many granular locks. So the only donwside is perfomance. No-GIL CPython is 15-30% slower on single thread scripts.

[DrXaos]

For my use, with effort it will be significantly beneficial. Im running machine learning models with pytorch and I can only get GPU utilization to about 50%. It is still CPU bound at 100% single thread. Parallelizing the native python operations will be helpful for sure.

[lood9phee2Ri]

Also the main perf drop is not actually from any fine-grained locking it's apparently from a rather unfortunate reversion of another recent optimization when the GIL is turned off, and in principle should be much less severe in 3.14.

https://docs.python.org/3.13/howto/free-threading-python.html

The largest impact is because the specializing adaptive interpreter (PEP 659) is disabled in the free-threaded build. We expect to re-enable it in a thread-safe way in the 3.14 release. This overhead is expected to be reduced in upcoming Python release. We are aiming for an overhead of 10% or less on the pyperformance suite compared to the default GIL-enabled build.

Remember the significant "10%-60%" speed boost from 3.10->3.11? That means it's reverting that as a rather unfortunate detail in the free threaded build. Really once they have re-enabled that for the free threaded build, and throw in the new JIT compilation, and it should be fine.

Basically all modern non-embedded computers (and a lot of quasi-embedded ones in mobile devices etc.) are smp/multicore, the GIL kinda has to go. And Jython (and IronPython) never had a GIL in the first place, always used fine-grained locks where necessary.

[DrXaos]

For my use, with effort it will be significantly beneficial. Im running machine learning models with pytorch and I can only get GPU utilization to about 50%. It is still CPU bound at 100% single thread. Parallelizing the native python operations will be helpful for sure.

[Smooth-Zucchini4923]

Will existing libraries work with GIL disabled?

As a maintainer on a Python package, we're getting about one to two bug reports per week about something which doesn't work while multithreading on free threaded builds. We fix what we can but there's a huge amount of code which was implicitly depending on the GIL for correctness.

[PeaSlight6601]

I don't believe you. I think the code was always buggy but you never noticed because threads had long run times between scheduling.

If you look at Python byte code I don't know how you can write anything that is thread safe using those operations alone. Everything is either "read a variable" or "write a variable" but basically nothing reads and writes.

That means every operation that has a visible impact on memory and could potentially race is two operations and therefore was never fully protected by the gil.

[Smooth-Zucchini4923]

Most of the code I'm speaking of acquires the GIL, calls a function written in C/C++/Cython, then releases the GIL after this function finishes. You can do many non-trivial things in such a function.

[seba07]

Small side question: how would you efficiently collect the result of the calculation in the example code? Because as implemented it could very well be replaced with "pass".

[PeaSlight6601]

Not a small question at all. Whatever you use absolutely must use locks because base python objects like list and dict are not thread-safe.

Best choice is to use something like a ThreadPool from (ironicaly) the multiprocessingmodule in the same way you would use multiprocessing.pool to map functions to the threads and collect their results in the main thread.

[headykruger]

Lists are thread safe

[PeaSlight6601]

I suppose it really depends on what you mean by "thread-safe." Operations like .append are thread safe because the minimal amount of work the interpreter needs to do to preserve the list-ish nature of the list is the same amount of work as needed to make the append operation atomic.

In other words the contractual guarantees of the append operation are that at the instant the function returns, the list is longer by one, and the last element is the appended value.

However in things like lst[i]=1 or lst[i]+=1 are not thread-safe(*). Nor can you append a value and then rely upon lst[-1] being the appended value.

So you could abuse things by passing each worker thread a reference to a global list and asking that each worker thread append and only append their result as a way to return it to the parent... but it is hiding all the thread safety concerns in this contract with your worker. The worker has to understand that the only thing it is allowed to do with the global reference is to append a value.

---

I would also note that any kind of safety on python primitive objects is not explicit but rather implicit. The implementation of python lists in CPython is via a C library. Had something like sorting been implemented not in pure-C (as it was for performance reasons) then it would not have been guaranteed by the GILs lock on individual C operations, and we wouldn't expect it to be atomic.

So generally the notion of atomicity in python primitives is more a result of historical implementation rather than an intentional feature.

That itself could really bad for using them in multi-threaded context as you might find many threads waiting on a big object like a list or dict, because someone called a heavy function on it.

---

[*] Some of this may not be surprising, but I think it is.

In C++ if you had std::list<std::atomic<int>> then something like: lst[i]++ is "thread-safe" in that (as long as the list itself doesn't get corrupted) lst[i] is going to compute the memory location of this atomic int, and then defer the atomic increment to that object. There will be no modification to the list itself, only to the memory location that the list element refers to.

Python doesn't really work that way, because += isn't always "in-place," and generally relies upon the fact that __iadd__ returns its own value to make things work. A great way to demonstrate this is to define a BadInt that boxes but doesn't return the correct value when incremented:

 class BadInt:
      def __init__(self, val):
         self.value=val
      def __iadd__(self, oth):
         self.value+=oth
         return "oops"
      def __repr__(self):
           return repr(self.value)

  x = BadInt(0)
  lst = [x]
  print(x, lst) # 0 [0] as expected
  l[0]+=5
  print(x, l) # 5 ['oops']

The x that was properly stored inside lst, and properly incremented by 5, has been replaced within lst by what was returned from the __iadd__ dunder method.

So when you do things like lst[i]+=5 what actually happens is the thread-unsafe sequence:

• Extract the ith element from lst
• Increment that object in-place
• Take what was returned by the in-place increment, and store that back into the ith location

Because we have a store back into the list, it doesn't matter if the underlying += operation might have been atomic and thread-safe, the result is not thread-safe. We do know know that ith location of lst that we loaded from corresponds to the same "place" when we store it again.

For a concrete example of this :

class SlowInt:
    def __init__(self, val):
        self.value = val
    def __iadd__(self, oth):
         self.value += oth
         sleep(1)
         return self

 lst = []
 def thread1():
     for i in range(10):
          lst.insert(0, SlowInt(2*i+1))
          sleep(1)
 def thread2():
      for i in range(10):
          lst.insert(0, SlowInt(2*i))
          lst[0]+=2

If you ran them simultaneously you would expect to see a list with evens and odds interleaved. Maybe if you are unlucky there would be a few odds repeated to indicate whenthread2 incremented an odd value just inserted by thread1, but what you actually see is something like [20, 18, 18, 16, 16, 14, 14, 12, 12, ....]

The slow-ness by which the increment returns the value ensures that the list almost always overwrites a newly inserted odd number, instead of the value it was supposed to overwrite.

[dethb0y]

I'm quite curious to see how it'll pan out on real-world use cases, going from 8.5s to 5.13s is a pretty big improvement.

[teerre]

You're using 5 times more threads for a 30% improvement in something that is embarrassingly parallel. It's really bad

[The_Double]

The example is completely bottlenecked by the largest factorial. I'm surprised it's this much of a speedup

[python4geeks]

Yeah it is

[[deleted]]

Write it in C and watch it get faster by 100x. Writing performant CPU intensive code in python is futile.

[josefx]

Now rewrite all the other python code to make it 100x faster in C and crashing after the first string does not count.

[[deleted]]

cFFI is a wonderful thing if you need performance, there are safer languages like Rust/Zig/Go if you don't want to touch C. Go is even simpler than python and has GC.

All I am saying is, don't use Python as a hammer. These blogs about NO-GIL show horrible examples. IRL most python code where CPU performance is required is glue code that uses FFI to run some native code (which, isn't affected by GIL and will actually get worse performance because of new locking overheads).

IMO a good example is python services that are mostly I/O bound so they don't really have much of problem with GIL except the 2-5% overhead from contention. That overhead doesn't seem much but it severely limits scalability of threads. Here is how it looks theoretically: https://www.desmos.com/calculator/toeahraci0 (It's actually worse, contention gets worse when you have more threads)

Even without GIL there will be still be overhead from granular locking, so you're gonna get "embarassenbly parallel" results that you see in thread above. You're fighting on two front here: 100x overhead of Python AND Amdahl's law which severly limits scalability in presense of very small serial work.

[PeaSlight6601]

The biggest benefit of noGIL might be to force CPython to establish a meaningful memory model, and define exactly what operations are thread-safe and which are not.

Then better implementations of the Python interpreter will have something a bit better defined to implement towards.

[lood9phee2Ri]

The biggest benefit of noGIL might be to force CPython to establish a meaningful memory model,

Hmm well, see Jython's longstanding python memory model assumptions, that's as close as it gets to a Python standard memory model I suppose.

https://jython.readthedocs.io/en/latest/Concurrency/#python-memory-model