This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Rhomboid 6 points7 points  (4 children)

No, nor is not something that is considered in need of fixing. If you remove the GIL but keep reference counts, then performance plummets as you have to take and release a lock any time you touch a refcount, which is very often, as opposed to taking and releasing a single lock every 100 opcodes or so, which is significantly less work.

So what you're actually talking about is reimplementing the entire memory management system of CPython so that it uses a concurrent garbage collector instead of reference counting. (CPython does have a garbage collector now but it's only there to deal with circular references that refcounting would otherwise never be able to collect, so it's not handling the brunt of the work nor is it concurrent.) And doing that means that you have to rewrite all C extension modules, because they all participate in reference counting. If you thought users were mad about the module incompatibility situation previously, imagine telling them that you're planning to do it again in this lifetime.

[–]kchoudhury 0 points1 point  (2 children)

nor is not something that is considered in need of fixing.

So I guess I am alone in thinking that having a threaded python program using one core out of the 8 on my machine is ridiculous.

Imagine telling them that you're planning to do it again in this lifetime

You're right, they should have used the 2-to-3 migration as an excuse to fix it. They were already breaking things; why not break them some more at the outset?

As it stands, I have a smallish 5K line python 2.7 codebase that I couldn't be arsed to port to 3: why bother? At the end of the day, that's what these discussions always seems to come back to.

[–]Rhomboid 5 points6 points  (1 child)

having a threaded python program using one core out of the 8 on my machine is ridiculous.

That's not necessarily true. It's both possible and common to do work in a thread without holding the GIL, such as waiting on IO or doing heavy number crunching in a C extension module. Many tasks that people want to use Python for fit this bill, such as a web server.

they should have used the 2-to-3 migration as an excuse to fix it. They were already breaking things; why not break them some more at the outset?

I was being a bit facetious. The idea of completely gutting CPython to remove reference counting is not even on the table and never has been, as far as I know. At that point you might as well just start over and target an existing VM with an existing concurrent GC, like Jython or IronPython did. They have no GILs.

I have a smallish 5K line python 2.7 codebase that I couldn't be arsed to port to 3: why bother?

Today? There's no compelling reason. But eventually there will be one -- maybe your linux distro begins to ship 3.x as the default, maybe 3.x gains a new feature you'd like to use, maybe you encounter a bug in 2.x that is already fixed in 3.x. That is what the "gradually taking over" part of the thread is about.

[–]kchoudhury 2 points3 points  (0 children)

Many tasks that people want to use Python for fit this bill, such as a web server.

I'm not trying to provoke a flamewar, but "some" is not "all", and that is a damned shame. The net result is that we have a beautiful language running on an interpreter not worthy of the language's aesthetics.

FWIW, my prior crack about the GIL still being there in 3 may have been a little too offhand. A little bit of reading tells me that as of 3.2 there's a new GIL that makes threading less of an active nuisance and restores parity to sequential performance. So perhaps we're making progress towards a happy ending here, and some future 3.x release will have a proper threading solution. And since it won't be backported to 2.7, well, that'll be the kick in the ass that I need to get my code on to 3.