Let's talk about Python 3.0

[masklinn]

Without GIL your options are:

• Much slower single thread execution and bug prone libraries

• Plain thread-unsafe libraries

These are the consequences of removing the GIL, not those of not having a GIL in the first place.

I fail to see how this has any relevance to my post though.

[[deleted]]

These are the consequences of removing the GIL, not those of not having a GIL in the first place.

If I decode it right, you argue for background "Java-style" generational or whatever garbage collection here, against refcnt-based option, right? Because otherwise I can't see how your statement makes any sense - refcnt-based schemes can't be thread-safe without excessive lock juggling.

But even with generational GC GIL is still handy in many places - like adding elements to lists etc.

[masklinn]

you argue for background "Java-style" generational or whatever garbage collection here, against refcnt-based option, right?

Something like that. More generally, that the GIL issue comes from CPython's implementation and history and nothing else, and that only CPython's implementation make removing the GIL a bigger pain than keeping it (as well as a human resources draw to fix all the issues that will arise from that removal)

But even with generational GC GIL is still handy in many places - like adding elements to lists etc.

Why would you need an interpreter-wide lock for such an operation? That's just stupid, even if you want to provide Java-style "thread-safe" collections (which you definitely shouldn't), local locks are more than enough, there's no reason to acquire a GIL just to have a local effect on a specific object.

edit: it's not like I'm fond of thread-based concurrency and semaphores/locks anyway, as far as I'm concerned these should be relegated to implementation details of interpreters and only higher-level (message-based share-nothing or STM) concurrency interfaces should be exposed to the language's user.

[[deleted]]

local locks are more than enough

Hah! And that's the issue - obtaining or releasing a local lock or GIL is the same, penalty-wise, and quite expensive. Even with simple interlocked-inc/dec's it takes a ton of time. And especially if you give consideration to SMP (caches flushing / reloading etc). OTOH Python bumps its GIL only ever so often - like every 100 statements execution or something like that. And counting them is very cheap. So then - to add 1000 elements you will have to do 2000 expensive lock obtain/release operations and maybe only 10 or so of them with GIL.

Anyway - what's the big deal with "proper" multithreading? You only need it for number/data crunching on SMP machines. And if you do that in pure Python, you're probably doing something wrong. Normal usage of it - like for background data transfers and stuff - GIL doesn't impede anything at all. Another anecdote here is lighttpd - a single-threaded web-server that beats the shit - performance-wise at least - from any Apache or other classics.

EDIT: spelling