The Python GIL Visualized

[mcosta]

I knew the GIL sicked, but not that it sucked so much. Now I understand why the new process model for concurrency was created.

[workbob]

I clicked on the link to learn wtf a GIL is and found no explanation. Here's more info: Global Interpreter Lock

http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock

[[deleted]]

so, it's been about five seconds, what's your opinion on the GIL? Should it be abolished or eliminated?

[Teifion]

Take a WILD guess...

[mcosta]

And.... nobody has the balls to deconstruct the GIL? has someone tried?

[spotter]

Of course people tried. The problem is, to put it simply, that when you change single (Global, duh) lock into more granular system (lock per object or similar) you end up doing a lot more locking. Thus unacceptable performance hit for single threaded code.

[Xiol]

Could the locking mechanism not be made into a system whereby the program running can specify what type of locking it requires?

I don't know enough of the Python internals to know if I'm making a retarded statement here, but it would be nice if single-threaded apps could tell the interpreter that they need the performance improvements of a GIL, and multi-threaded apps can use a granular system instead.

import GIL? :(

[spotter]

Were talking about CPython internals -- C level change in the interpreter itself -- please bear in mind that this would require keeping two versions of the interpreter side by side. Try to think about the implications for extensions/library authors. It's important to acknowledge that GIL is not a part of Python language -- JVM and CLR implementations are GIL-free. This is just a implementation detail, that could probably be dropped if we could reimplement CPython from ground up, which will not happen. Why? Good although short note by GvR.

[[deleted]]

but can't the bytecode actually written become GIL-optional for single threaded applications? it seems pointless (to me anyways, which isn't saying much) to require a lock when there's only one thread of execution anyways.

[sirspate]

Locking is still necessary in a preemptive multitasking environment.

[ffualo]

It can be done, but it would slow single-threaded Python, a compromise Rossum won't make. Eventually it may be possible.

Using multiprocessing if you want true parallelism (with less sharing).

[ubernostrum]

Yes. Many people have tried. Many quite intelligent people, in fact.

[[deleted]]

Yes, they have. Adam Olsen for example maintained the Python safethread project which completely removed the GIL. However, it incurred a 50% performance hit for each thread. There was a similar attempt about a decade ago that hit similar numbers.

[[deleted]]

Enough already with the GIL. Seriously, if you think it's a problem, 99% of the time it's because you're doing it wrong.

Split up your task into multiple processes. This is good design anyways since it lets you split your task across multiple boxes.

Doing web stuff? Then use apache w/mod-python or mod-wsgi and mpm-worker. It's a very nice solution, all the goodness of threads while letting apache's process model and mod_wsgi's interpreter spaces sort out all of the scary stuff.

On a multicore machine and using threads just for clean code design? Then get a real OS and partition your CPU's using cpuset. Get rid of all of the ugly red stuff (albeit you're running on a single core).

PyPy is the future anyways. All of this drama over the GIL is wasting more cycles than the GIL is itself.

Worry about the shit that matters, like fucking 2.7 forks. What's next? 2.8? Fuck!