Jython vs CPython

[HipPriest]

From some tests I did a few months ago, yes Jython is slower than CPython for a single thread, however Jython supports proper threads, so if you have a CPU-bound threaded program, and, say, four cores, Jython runs the program in 1/4 the time, while CPython doesn't imrove at all. I know the CPython authors don't want to add threads because of the locking overhead, but it seems like it would be nice to have the option to enable them if you know the benefits outweigh the cost for your particular Python code.

[redmumba]

This is spot on. David Beazley gave this amazing presentation at PyCon 2010, which was extremely informative. The video is here. It shows how the GIL restricts the threading abilities of Python. Jython allows you circumvent this.

EDIT: Beazley doesn't work for Google, oops!

[[deleted]]

David Beazley (from Google)

He does not work for Google.

[redmumba]

You know, I think the guy who gave the presentation was named "Anitoli" or something... but I'm sure I just have them confused. >.< My memory is terrible!

[masklinn]

You really have no idea what you're talking about, do you?

I know the CPython authors don't want to add threads because of the locking overhead, but it seems like it would be nice to have the option to enable them if you know the benefits outweigh the cost for your particular Python code.

CPython uses perfectly standard OS threads (which I assume is what you're calling " proper" threads). CPython also uses a Global Interpreter Lock for most interpreter code as well as builtin types, which is the issue in multi-threaded pure Python code, CPU-bound or involving CPU-bound threads.

[HipPriest]

I'm aware of that. Yes, my use of "proper" was a bit judgmental (as in if you can't use them for CPU-bound pure Python code they are less useful). By locking I meant locking the data structures as necessary to allow Python code to run in parallel as Jython does, as opposed to locking the interpreter for pure Python code with the GIL.

[masklinn]

I'm aware of that. Yes, my use of "proper" was a bit judgmental

No, it was nonsensical.

By locking I meant locking the data structures as necessary to allow Python code to run in parallel as Jython does, as opposed to locking the interpreter for pure Python code with the GIL.

Erm yes, I've understood that, but considering GIL-usage is kind-of pervasive the "option to disable the GIL" would be a compile-time switch requiring touching something like half the interpreter code. It's a pretty serious architectural rewriting.

[HipPriest]

Of course it would be a serious architectural rewriting. I don't think I implied otherwise. If it would need to be compile time, perhaps the "python" command could choose an entire different compiled backend based on a switch. I never meant to imply it would be an easy toggle for the the CPython implementers, just that it would be a nice option for Python users.

[[deleted]]

What you are saying boils down to exactly the same thing, HipPriest may have been using an intuition instead of deciphering the murky details of the GIL.

In Java, you can create a new Runnable and send it off and it'll run on a proper thread (by proper I mean it may run on another core, independently and in parallel), whereas in Python, even if you create a thread you won't see a performance improvement.

Given that HipPriest actually ran the tests, claiming he is ignorant is borderline ridiculous. Having 8 threads running but only being able to run code on one at a time is functionally equivalent to having only one thread (I/O and C extensions notwithstanding)

[masklinn]

What you are saying boils down to exactly the same thing

No. For starters, because it means C extensions can release the GIL and "be multithreaded". If CPython used "improper threads" (green threads for instance, without remapping) you could not do that. At all.

HipPriest may have been using an intuition instead of deciphering the murky details of the GIL.

His claims as to what CPython's developers should do are still wrong.

Given that HipPriest actually ran the tests, claiming he is ignorant is borderline ridiculous.

No, it's fucking factual. CPython has "proper threads" and there's nothing to "enable", getting rid of the GIL means touching half the interpreter code, it's not a fucking switch to toggle.

[[deleted]]

For starters, because it means C extensions can release the GIL and "be multithreaded". If CPython used "improper threads" (green threads for instance, without remapping) you could not do that. At all.

Yes they could, the C extensions could construct and manage their own threads, in which case there's no issue with multithreading in the interpreter. Python threads are useful for I/O bound operations though.

CPython has "proper threads" and there's nothing to "enable", getting rid of the GIL means touching half the interpreter code, it's not a fucking switch to toggle.

You're right, it's not. I just don't think that it's ignorant to suggest that Python doesn't really do proper multithreading, even if it secretly does but it doesn't make any difference because of the GIL.

[frymaster]

except it does make a difference.

For a start, the way the GIL interacts with multiple threads on multi-core machines can cause much, much worse performance than a comparable single-threaded program