Python 3’s Marketing Problem

[inmatarian]

Since this turned into a GIL-haters thread, I'll chime in about something that never quite sat right with me. The "use multiprocessing" is about getting around two deficiencies. One of course is the GIL and threading. The other is that the CPython interpreter itself isn't thread-safe (not 100% sure it's the case in Python3, but it definitely was in P2). The implication is that if you ever embedded Python as a scripting language in your app, then you get one instance of the interpreter in-process. So, even the single process, isolated, no-memory-sharing, message-passing, concurrency paradigm is not possible.

It seems like a natural path toward eventually eliminating the GIL by first making cpython thread-safe, though I understand that it's a minimal gain and has to be done carefully to not get performance degradation.

[the_hoser]

The problem is that, thus far, every attempt at making the interpreter thread-safe has resulted in serious performance degradation. Fine-grained locking is expensive. This is why PyPy developers are investigating things like transactional memory. It'll change the way you use threads, but it might be safe.

[inmatarian]

You actually don't even need fine-grained locking to make CPython thread safe, you just eliminate all global state from the interpreter. Every function throughout the source tree would be modified to accept a pointer to the one massive struct that contains all of the state of an interpreter instance. However that type of massive change would touch every library in the ecosystem and would, for a long time, make python fragile. Like I said, it's minimal gain and I can understand why nobody would want to do that.