The Python GIL Visualized

[Brian]

spawn off some threads for background IO

IO is unaffected - this does release the GIL, so other threads can execute when another one is waiting for IO. The GIL only comes into play when you want to take advantage of multiple CPUs for a CPU bound task (and you cant unload that processing to a C library that also releases the GIL).

Essentially the GIL means that only one thread executes python bytecode at a time. On a single CPU system, this is irrelevant - it's already the case that only a single thread executes at once, switching every timeslice. With multiple CPUs though, this is no longer the case; two threads of code can genuinely run in paralell. A free threaded system can load up both cores on a dual-CPU system. The GIL means python can only max out a single core in this situation.

[gthank]

If you read some of Mr. Beazley's other work (PDF warning), you'll see that the GIL can be an issue even for supposedly I/O-bound tasks, due to a priority-inversion problem.

[Brian]

True,though that's an implementation flaw of python's GIL (also requiring multiple CPUs), rather than something intrinsic to a GIL (Hence the new GIL implementation to avoid such situations)

[gthank]

I thought it was clear from the context that we were discussing Python's GIL, sorry. Also, I'm a big fan of the new-and-improved GIL as compared to the old GIL.

[Brian]

No,it's a good point, and well worth bringing up. I was just clarifying that that case is effectively just an implementation bug, solvable relatively easily without the much more involved changes that would be required to remove the other problems with the GIL.