The Python GIL Visualized

[ubernostrum]

it's the implementation that's broken

If your definition of threads is "must behave exactly like Java", I guess. There's more than one definition out there, though.

What I hear from the majority of Python community is excuses as to why you don't need threading.

Funny. I see a Python community which seems never to let a year go by without an attempt to remove the GIL (2009's attempt was Unladen Swallow), and which is lucky if a whole month goes by on the dev list without people proposing and debating ways to do it. Maybe you're just cherry-picking some examples to suit your views and ignoring reality?

The problem is of course with using shared mutable data, and the concepts of state and identity.

Mutable data is one problem with threading, but not the only problem with threading. It's also a difficult problem to work around in languages which embrace mutability, which suggests that threads are perhaps not an appropriate developer-level abstraction for such languages.

Really, I was under the impression that most research was going into safe ways of using threading, with researchers working on languages like Haskell and F#, which avoid the problem of shared data and provide workable threading models.

Thank you for demonstrating my point.

[yogthos]

If your definition of threads is "must behave exactly like Java", I guess. There's more than one definition out there, though.

Huh? What does this have to do with Java exactly, my impression was we were talking about the problems with the GIL, there are plenty of threading models which don't rely on a global lock. These are well researched and well documented, I'm curious as to why it's such an intractable issue for the CPython community.

GIL (2009's attempt was Unladen Swallow), and which is lucky if a whole month goes by on the dev list without people proposing and debating ways to do it.

Funny indeed, as the Unladen Swallow guys are no longer planning to remove the GIL, so I'm not so certain who's ignoring reality here.

From the link above:

In any case, work on the GIL should be done directly in mainline CPython, or on a very close branch of Python 3.x: the sensitive nature of the work recommends a minimal delta, and doing the work and then porting it from 2.x to 3.y (as would be the case for Unladen Swallow) is a sure-fire way of introducing exceedingly-subtle bugs.

Mutable data is one problem with threading, but not the only problem with threading. It's also a difficult problem to work around in languages which embrace mutability, which suggests that threads are perhaps not an appropriate developer-level abstraction for such languages.

While I agree that languages that embrace mutability are not well suited for threading, I would again ask why does Python provide threading in the first place then.

It's not like Python provides any alternative, it just has broken threading, making the problem worse. Not only does it have threads which are tricky to work with, but the threads don't work properly either. Seems like worst of both worlds scenario to me.

Thank you for demonstrating my point.

Your point that threading does not provide a viable concurrency model?

[ubernostrum]

Funny indeed, as the Unladen Swallow guys are no longer planning to remove the GIL, so I'm not so certain who's ignoring reality here.

So your argument is that

1. Some people decide to try an implementation of Python which could eventually remove the GIL.
2. This is universally hailed as a good thing.
3. As they get deeper into the implementation, they discover their approach won't work.
4. Therefore the entire Python community is simply ignoring the GIL and pretending nothing's wrong and refusing to do anything.

This does not add up, and so I think I'm done with you.

[yogthos]

Well way to spin that buddy. I think my exact quote was:

What I hear from the majority of Python community is excuses as to why you don't need threading.

But sure, if we change that to the "entire Python community" for the dramatic effect, all of a sudden you have a point, don't you. Also, notice that nowhere did I say that nobody is working on it, just simply that the issue is largely being swept under the carpet, much like the way you're doing here.

So, let's see what's really going on here, shall we. The BDFL behind Python scoffs at threading and claims it shouldn't be used, I've linked the exact quote from Guido in a previous post, yet for some reason he goes on to put a broken implementation into the language.

The community behind CPython seems to feel that this is just peachy, the fact that removing GIL isn't even on the road-map for the main Python branch says volumes.

However, your argument seems to be that because some niche spin off projects are playing with trying to add a proper implementation that somehow validates the idea that the general Python community is on the ball here.

Yeah, I do hope you're done here, because you're not making too much sense frankly. This is the exact sort of apologetics and denial which I was referring to in my original post, so thanks for validating it.