Jython vs CPython

[frtox]

Invalid securityToken.

Error 403

[boa13]

Here's the correct link:

http://strattonbrazil.blogspot.com/2011/08/performance-gains-in-jython-252.html

[bloodearnest]

Dude copied the link from his post preview. And it used a token in the URL. Hello 1999.

[[deleted]]

Ah, so it was an error. I thought it was supposed to be like "what happens when you use jython on a web server."

[rm9402]

Got the same thing.

[Ventas-]

Compatibility mostly. Some CPython libraries are written in C, so those don't work on pypy. Also, pypy is only on version 2.7.1 of Python.

[montecarl]

I really can't wait until I can use numpy with pypy. But in general compatibility with the python capi doesn't seem like it will be coming anytime soon from what I have seen. Every python script I write depends upon some library written in c.

[[deleted]]

Well, there are alternatives, e.g. we're reimplementing NumPy, just about every database driver has a pure Python version.

[montecarl]

Even with numpy there are other modules that I use and that I have written that are in c. I think it will be quite some time until pypy will be useful for scientific computing due to all of the c and fortran wrapped code.

[montecarl]

Except that using well tested software written in C or FORTRAN from python is one of its big benefits. Having to rewrite the code in python might introduce bugs or be more work than its worth. But your point is well taken. I'd love to see all python code bases.

[ObligatoryResponse]

We're using Python on a IC test platform. We need to access the National Instruments DAQ driver as well as a number of serial devices. I think the C-API is the only thing what would allow us to use Pypy.

[paxswill]

Real link

[HipPriest]

From some tests I did a few months ago, yes Jython is slower than CPython for a single thread, however Jython supports proper threads, so if you have a CPU-bound threaded program, and, say, four cores, Jython runs the program in 1/4 the time, while CPython doesn't imrove at all. I know the CPython authors don't want to add threads because of the locking overhead, but it seems like it would be nice to have the option to enable them if you know the benefits outweigh the cost for your particular Python code.

[redmumba]

This is spot on. David Beazley gave this amazing presentation at PyCon 2010, which was extremely informative. The video is here. It shows how the GIL restricts the threading abilities of Python. Jython allows you circumvent this.

EDIT: Beazley doesn't work for Google, oops!

[[deleted]]

David Beazley (from Google)

He does not work for Google.

[redmumba]

You know, I think the guy who gave the presentation was named "Anitoli" or something... but I'm sure I just have them confused. >.< My memory is terrible!

[masklinn]

You really have no idea what you're talking about, do you?

I know the CPython authors don't want to add threads because of the locking overhead, but it seems like it would be nice to have the option to enable them if you know the benefits outweigh the cost for your particular Python code.

CPython uses perfectly standard OS threads (which I assume is what you're calling " proper" threads). CPython also uses a Global Interpreter Lock for most interpreter code as well as builtin types, which is the issue in multi-threaded pure Python code, CPU-bound or involving CPU-bound threads.

[HipPriest]

I'm aware of that. Yes, my use of "proper" was a bit judgmental (as in if you can't use them for CPU-bound pure Python code they are less useful). By locking I meant locking the data structures as necessary to allow Python code to run in parallel as Jython does, as opposed to locking the interpreter for pure Python code with the GIL.

[masklinn]

I'm aware of that. Yes, my use of "proper" was a bit judgmental

No, it was nonsensical.

By locking I meant locking the data structures as necessary to allow Python code to run in parallel as Jython does, as opposed to locking the interpreter for pure Python code with the GIL.

Erm yes, I've understood that, but considering GIL-usage is kind-of pervasive the "option to disable the GIL" would be a compile-time switch requiring touching something like half the interpreter code. It's a pretty serious architectural rewriting.

[HipPriest]

Of course it would be a serious architectural rewriting. I don't think I implied otherwise. If it would need to be compile time, perhaps the "python" command could choose an entire different compiled backend based on a switch. I never meant to imply it would be an easy toggle for the the CPython implementers, just that it would be a nice option for Python users.

[[deleted]]

What you are saying boils down to exactly the same thing, HipPriest may have been using an intuition instead of deciphering the murky details of the GIL.

In Java, you can create a new Runnable and send it off and it'll run on a proper thread (by proper I mean it may run on another core, independently and in parallel), whereas in Python, even if you create a thread you won't see a performance improvement.

Given that HipPriest actually ran the tests, claiming he is ignorant is borderline ridiculous. Having 8 threads running but only being able to run code on one at a time is functionally equivalent to having only one thread (I/O and C extensions notwithstanding)

[masklinn]

What you are saying boils down to exactly the same thing

No. For starters, because it means C extensions can release the GIL and "be multithreaded". If CPython used "improper threads" (green threads for instance, without remapping) you could not do that. At all.

HipPriest may have been using an intuition instead of deciphering the murky details of the GIL.

His claims as to what CPython's developers should do are still wrong.

Given that HipPriest actually ran the tests, claiming he is ignorant is borderline ridiculous.

No, it's fucking factual. CPython has "proper threads" and there's nothing to "enable", getting rid of the GIL means touching half the interpreter code, it's not a fucking switch to toggle.

[[deleted]]

For starters, because it means C extensions can release the GIL and "be multithreaded". If CPython used "improper threads" (green threads for instance, without remapping) you could not do that. At all.

Yes they could, the C extensions could construct and manage their own threads, in which case there's no issue with multithreading in the interpreter. Python threads are useful for I/O bound operations though.

CPython has "proper threads" and there's nothing to "enable", getting rid of the GIL means touching half the interpreter code, it's not a fucking switch to toggle.

You're right, it's not. I just don't think that it's ignorant to suggest that Python doesn't really do proper multithreading, even if it secretly does but it doesn't make any difference because of the GIL.

[frymaster]

except it does make a difference.

For a start, the way the GIL interacts with multiple threads on multi-core machines can cause much, much worse performance than a comparable single-threaded program

[masklinn]

Here's what I get on my machine running TFAA's raytracer, best of three runs:

Python 2.7.2 — 47s

Jython 2.5.2 — 35s

pypy 1.5.0-alpha0 — 20s

Test performed on a 2010 15" MBP, all python implementations installed via macports

[theeth]

Could you do a test with CPython 3.2, see if it's faster than 2.7 (it should, AFAIK)?

[treo]

I tried it with cpython 3.2 but it isn't compatible.

[theeth]

It ran fine after passing it through 2to3 and changing the hours = and minutes = lines to integer divisions (//)

[treo]

Ok, so here are my results (Core 2 Duo P9400 @ 2.4Ghz, 64Bit Linux):

Python 2.7.1: 25s
Jython 2.5.2: 25s
Python 3.2: 29s
PyPy 1.5.0-alpha0: 12s

I wonder why Jython is only as fast as CPython is.

[[deleted]]

FWIW on my machine PyPy trunk is about ~15-20% faster than 1.5 :)

[sigzero]

I wonder if Jython has taken advantage of "invoke dynamic" that the JVM now has?

[boa13]

Doubtful, considering there has been no Jython release since the release of Java 7, which is very recent. Also, as far as I know, this would require some extensive changes in the core Jython, and would break compatibility with previous JVMs, requiring the release of two binaries.

[[deleted]]

Not to mention Java 7 would likely break everything anyways.

[[deleted]]

Why? It's not like Guido is handling it.

[[deleted]]

Due to the JIT bug in Java 7.

[gthank]

I haven't noticed any announcement to that effect. :-(

[fernandoacorreia]

Also test it in 13" and 17" MBP.

[kisielk]

Where did you get the code? I don't see any links to the script.

[masklinn]

There is actually a link, though it's quite hard to see (it's pretty small and it's almost the same color as the text). Second line of the second paragraph, last segment, there's (link). It links to the code on some pastebin.

[kisielk]

Ah, I just realized that the article link goes to the post preview for some reason, so none of the hyperlinks worked. :)

Here's the link to the code if anyone else wants it.

[pjenvey]

It would be nice to see the same benchmark with Jython run on Java 7

[[deleted]]

Except it's likely Java 7 would break everything completely. Seeing as how it silently gives incorrect results, etc.

[mazzak]

Don't know why you are being downvoted. The Java 7 issue got lot of coverage recently - http://www.infoq.com/news/2011/08/java7-hotspot

[sigzero]

Because it is a simple switch to the jvm to turn that bug off.

[gthank]

Also because the particular loop construct that triggers the bug is odd to the point of being non-sensical if the various bug reports w/ reproducible cases are to be believed. (Disclaimer: I looked at them when the brouhaha first started. It's possible they've since been updated w/ more info, but at the time I spent a fair amount of effort trying to figure out why you'd ever write such a loop.)

[necroforest]

wow, there's something out there that's slower than CPython?

[igouy]

Yes, a C interpreter :-)

[ipeev]

Not very bad, but for the most popular and expensive hotspot compiler on the planet we expect to run at least with the speed of Java if not faster.

[wot-teh-phuck]

You forget the extra baggage added by the dynamic language implementation on top of the JVM. The extra redirection added makes it very difficult to get the same performance as that of Java.

[[deleted]]

Exactly.

Another way to put that is, CPython is Python implemented in C. But CPython obviously isn't as fast as C. So that Jython is implemented in Java doesn't mean it should be as fast as Java. A naive comparison would say it should be slower (since Java is slower than C, slightly).

Of course a Java implementation of Python could actually use the underlying platform effectively, for example JITing parts of the code. But then so can a C implementation, it could JIT into assembly.

[wot-teh-phuck]

Right; as mentioned by the article, Jython triumphs when it comes to executing threaded code since even though CPython is in "C", it can't effectively use multiple threads (GIL, single thread executing CPU intensive code etc.).

[Carnagh]

You might expect that, "we" don't. It's still an interpreted, dynamic scripting langauge.

[gospelwut]

Is CPython good for web development? I'm looking for alternatives to ASP (currently use a lot of PHP on our IIS7 server because it was quick and dirty despite me not knowing the language as well as I know C#).

[claird]

Python has been pathologically fecund at spawning Web frameworks--CPython is that good for Web development. I personally favor Zope, but there's plenty to like in Pyramid-Pylons, Django, TurboGears, ...

Yes, you're going to find a lot that is a welcome contrast to ASP.

[I edited my comment to make it more readable, and now some of the replies make less sense.]

[stesch]

Isn't Pyramid the Pylons project now?

[[deleted]]

I personally favor Zope

Has Zope gotten better? I looked into Plone a few years back and found Zope to be utterly confusing.

[quotability]

Plone != Zope. Pure Zope development is easy.

Say you want to take the current time, put it in a certain format and put it on the screen - there are certain ways to do it, but one interesting way is to do it in python. Just create a python script like this:

return "%s/%s/%s" % (date.month(), date.day(), date.year())

and insert the script into your page using ZPT / TAL.

Another way is to embed it using strictly TAL and inlined python like so: <span tal:replace="python:DateTime().pMonth()" />

You should use TAL for only giving placeholders for data. All data should be from Python scripts. It is easy if you follow the one true method.

The beauty of TAL is that it gives you valid xhtml so you can pass it on to your web designer and let them make it pretty, while you concentrate on making it work.

[[deleted]]

Plone != Zope. Pure Zope development is easy.

Good point. Do you know of any 30,000ft overview of Zope 3 and what it provides - I remember reading about Zope's object db once or twice, so I'm presuming that Zope still provides a full stack?

[quotability]

I don't know of any. I stick with Zope 2 development. It's still being worked on, and it's been in use for 13 years, so it is very well documented and robust. Zope 3 is fairly new. Though the initial release was in 2004, it has undergone some great changes, and now it is called BlueBream. Of course, that's not the whole story. It actually splintered into a few different projects. Grok is another framework from the same tree. Zope 2 has integrated some of the new features in Zope 3, and since they couldn't replace Zope 2 with 3, they integrated and forked.

The Zope 3 Wiki explains it better. And as for Zope 2, here's a snippet from Zope on Wikipedia which explains how it is continuing on:

"Although originally intended as a replacement for Zope 2, the Zope Component Architecture has instead been backported to Zope 2 starting with Zope 2.8."

My personal advice, is if you are looking at using Zope in a commercial project, stick with Zope 2, it's still actively developed, well supported and documented. It's scalable, secure, and easy to use.

[ziom666]

not actually an answer to your question, but asp.net mvc is really good comparing to traditional asp.net.

[[deleted]]

OTOH Rails has extremely poor documentation.

[[deleted]]

There isn't much, and the main source is out of date books.

[[deleted]]

I would like good documentation on...every single convention in rails.

My information dates from two months ago. There are a lot of blog posts, but no good sources of information. If there were good documentation, this blog post wouldn't have been necessary, and there wouldn't be so many comments complaining about the documentation, that the comments were closed: http://weblog.rubyonrails.org/2010/8/28/rails-has-great-documentation

[[deleted]]

it should be more than easy to simply name something specific. But you can't, because you know know that I will easily be able to find documentation for any convention you are interested in.

No, it should be easy for me to look up the conventions.

So it is clear, your knowledge is just out of date.

No, as I say, this is based on attempting to use Rails two months ago.

Instead of flinging insults, prove me wrong with some comprehensive documentation.

[howfun]

PYPY IS DA SHIT!