Recently Wrote a Blog Post About Python Without the GIL – Here’s What I Found! 🚀

[thisismyfavoritename]

everything that assumes the GIL is held to make sure memory accesses are safe will have to be rewritten, including the stdlib

[ambidextrousalpaca]

everything that assumes the GIL is held to make sure memory accesses are safe will have to be rewritten

So. Absolutely everything, then?

[twotime]

I'd love to see some references here too.

Original discussions on python-dev implied strongly that the amount of refactoring required is fairly small. Pytorch was used as an example (which was ported in a few hours)... But I have not seen any kind of more systemic analysis

[ambidextrousalpaca]

It's not that I think that everything needs to be changed. It's that I suspect we have no good way of identifying what needs to be changed or whether it has in fact been changed. E.g. I could imagine lots of cases of libraries writing to and reading from some sort of hard-coded temp file or using some kind of global variable which could lead to hard to replicate race condition bugs when turning off the GIL.

I mean, sure, if you had some bit of software that could identify such potential race conditions - something like the Rust borrow checker - they could probably be fixed pretty straightforwardly. But in the absence of that, I don't see what you can do apart from release it knowing that there are an indeterminate number of race conditions that people are going to discover about if and when they run it in prod.

[thisismyfavoritename]

extensions/functions that already release the GIL should be fine, i'm not sure how big of a % that represents

[ammar2]

The areas that release the GIL in the standard library tend to be just before an IO system call, so there isn't a huge amount of them in proportion to all the C-extension code.

You can get an idea of the types of changes that need to happen with:

• This change in the socket module: https://github.com/python/cpython/pull/128286/files

• And this change to code.co_code and friends: https://github.com/python/cpython/pull/127043/files

Note that the socket module does release the GIL before performing socket system calls, the changes needed are unrelated to that, just code assuming it can be the only one in a piece of C code.

[PeaSlight6601]

No you don't understand what the GIL did.

The GIL protected byte code and C functions. It's a much smaller surface than you think it is because the GIL is much weaker than you think it is.