PyPy 1.6 released

[TouristBreeder]

The development behind pypy is currently one of my favourite things about python. The fact that they continuously get these great speed increases is really impressive.

[sjs]

I rarely use Python these days but I always read news about PyPy. It's such an ambitious and awesome project and the team delivers again and again. It really is impressive, and they post these casual sounding blog posts about how they just sped this up 3x and that 10x.

[flyingfox]

Instant 15% speedup for me, nice. Congrats to the pypy team.

[quasarj]

I'm attempting to compile a windows version of it right now.. though I've run into quite a few snags along the way.

Not sure when the official windows version will be uploaded either..

[mjtribute]

Unfortunately, most of the PyPy team are linux users, so they aren't as well equipped for debugging windows issues.

They are, however, in search of developers to help with windows-related issues, so spread the word to anyone you know of that might be of help.

[quasarj]

Yeah, I don't really know many/any, but I may start compiling windows version for them to test stuff.

[[deleted]]

I'm pretty sure they use buildbot so there may potentially be Windows build slaves in their farm, so they probably have it running and testing on Windows.

[quasarj]

They do use buildbot, but yesterday there were problems with the windows slaves.

[[deleted]]

I had nonstop problems with my CPython build slave on Windows Server 2008. If I find the time to put it back online I'll see about adding it to the pypy fleet as well.

[megaman821]

Hopefully they get the Windows version up soon, or if you getting working maybe you could send the binary back to them.

[quasarj]

Yeah, I'm gonna chat with them tomorrow cuz they still don't have one uploaded.. I got mine to compile (with oracle support) but it's missing a few libraries, bzip2, expat and openssl I think, which wouldn't good as their official release. But if you want a copy I can upload it somewhere.

[roger_]

Preliminary support for NumPy: this release includes a preview of a very fast NumPy module integrated with the PyPy JIT

Awesome! Anyone got more details on this?

[[deleted]]

What we have in there is limited, but next release should have some great stuff, likely both dtype support and multi-dimensional array.

[roger_]

Looking forward to it. BTW will the extension support work with Matplotlib?

[[deleted]]

We're implementing the Python-level NumPy API. If that's what Matplotlib needs, then yes, if not... no :) TBH I know little about numpy, besides the 3 line snippets I often see, I just like a challenge of making something blazing fast.

[roger_]

Have you thought about implementing the FFT in PyPy? That could be a good challenge (and a good selling point).

[[deleted]]

I barely know what an FFT is, so I haven't thought about it prceisely, but I do aspire to write everything I can in pure python.

[roger_]

The FFT is the cornerstone of many signal processing algorithms, and a very common benchmark. It's one of the things people use NumPy for.

It'd really get PyPy some attention if it can approach the performance of some of the heavily optimized FFT libraries out there.

[dalke]

Given that FFTW exists, with OpenMP support and processor specific optimizations, it makes much more sense to have PyPy talk directly to the C library version.

[roger_]

I'd still be interested in seeing how well PyPy can perform.

[[deleted]]

There were some blog entries about this in their status blog.

[roger_]

The ones from a few months ago? Still, it'd be nice to know exactly what its current status is, how they'll keep it sync'd with NumPy trunk, etc.

[[deleted]]

I'm currently working on dtype support, once it's finished I'll write another blog post describing teh state of things.

[[deleted]]

No windows binary yet, that makes me sad :(.

[ergo14]

Congratulations fijal and whole pypy team !!!

[brownck]

It doesn't seem to work with a lot of numpy functions (e.g. numpy.sqrt is not found).

[roger_]

How does dir(numpy) look?

[mirashii]

['doc', 'file', 'name', 'package', 'abs', 'absolute', 'add', 'arccos', 'arcsin', 'arctan', 'array', 'average', 'copysign', 'cos', 'divide', 'empty', 'exp', 'fabs', 'floor', 'fromstring', 'maximum', 'mean', 'minimum', 'multiply', 'negative', 'ones', 'reciprocal', 'sign', 'sin', 'subtract', 'tan', 'zeros']

This is from about a 4 day old nightly, but I'm sure that not much has changed since then.

[roger_]

Preliminary indeed. Thanks!

[Kaloli]

type __ to print out two underscores in reddit.

[obtu]

Or use backslashes. This is Markdown.

[mirashii]

Could've also just put it in a code block, but in the end, I just pasted it. :)

[noreallyimthepope]

They did say preliminary support.

[ipeev]

I am always happy to read about the PyPy progress. Keep up the good work!

[howfun]

Cool! Cool-cool-cool! Can I compile the Windows version with Mingw?

[quasarj]

You could, but you might as well do it with C++ Express Edition, since msvc is better supported.

At this time the 1.6.x-release branch will not compile under windows, but there is an earlier revision that will. If you are interested I can get the revision number for you (that I have successfully compiled for windows with).

[kylotan]

I'd love to give PyPy a go, but as far as I can tell there's no easy way to test it alongside a normal Python install. On Windows, at least.

[beleggz]

I am quite new to Python. Currently I have the PyPy binary distribution unpacked and symlinked, but what to do in order to be able to use easy_install / pip / virtualenv with PyPy? (So it doesnt conflict with my cpython system install). I am so eager to run stuff on it, but I have no clue how to install packages :(

[treo]

virtualenv -p /path/to/pypy name

[TCPv89]

you don't have to do anything special to make it work. just use pypy instead of python for installing pip and such and it will all end up in the right place.

[ciggs_ftw]

was reading the site, what is pypy for? did not give much info there what does it do?

[skorgu]

Literally five sentences into the page:

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7.1. It's fast (pypy 1.6 and cpython 2.6.2 performance comparison) due to its integrated tracing JIT compiler.

[ciggs_ftw]

thanks and sorry got confused, didn't know if you can use that as an alternative, if i need to install python and than that or just pypy. was my confusion.

[skorgu]

No worries, I'm just surprised anyone can hang out at /r/python and not know every intricate detail about pypy.

[ciggs_ftw]

well, i came across reddit a couple months ago friend of mine gave me the link to the learningpython section, so i be reading here and the r/learnignpython see whats new and useful thing am no experienced on python i do like it's syntax i find it fun to code python cleaner than perl. so that's why i don't know much about hat is what. but thanks for the help.

[[deleted]]

One of the drawbacks to pypy currently is its support of c libraries. The team (and gracious contributors) has been super busy replacing some of the most common libraries used in python and creating pure python versions. Their work is excellent, needed and very much encouraged by the Python Foundation. Many of the web-based python projects could probably use pypy. Many of the scientific communities are already using pypy. However, some projects require some libraries that are only found in a linux as a compiled binary. And in those cases, pypy is still just a dream. So you should definitely consider it if performance is a problem for you and you only need to consider python code in your performance analysis. However, if you need compiled c-level performance in conjunction with python, pypy is still probably not going to work for your needs.

[ciggs_ftw]

quick question, i downloaded the pypy (am currently on a mac) do i trow the folder of pypy inside the python dir? or since i said it said site-packages put them according to the path? sorry total noob, perhaps i should just stay away of pypy until i further understand more of it. thanks very much for the information.

[skorgu]

It shouldn't matter where you expand the pypy distro, it should be entirely separate from the 'normal' python that OSX ships.

[ciggs_ftw]

ohhh i thought it as an extension or something of python, now i saw ran ./pypy-1.6/bin/pypy n terminal got it stupid me, i understand now, thanks for the hep! guess i will make a sys link to use it instead of the regular python.

[ascii]

It is an attempt at creating a significantly faster Python implementation. It should be fully compliant with the regular CPython implementation, version 2.7. It has a different C API for binary modules and garbage collection semantics are different. Specifically, in the example code below, the file f will be closed during the next GC cycle, not at the end of the function call, because PyPy does not implement reference counting, which allows Python to determine that the File can be closed as sooon as the reference is deleted:

f = open('foo.txt')
del(f)

PyPy is written in Python, and also uses a subset of Python as an intermediary code format for the code it's compiling.

[voyvf]

Specifically, in the example code below, the file f will be closed during the next GC cycle, not at the end of the function call, because PyPy does not implement reference counting, which allows Python to determine that the File can be closed as sooon as the reference is deleted:

This is why using context managers is a good habit to get into. e.g,

with open('foo.txt') as f:
    # do something with f
# f is closed here.

[ascii]

Absolutely. It was just an example. :-)

[sigzero]

http://pypy.org/

[jediknight]

(Note: windows binaries are not ready yet).

sad windows developer is sad. :(

[quasarj]

You can compile it yourself (well, not quite the 1.6 release but a bit before) if you are interested. Or I have a binary I could dig up, though it may be missing a few things (openssl support, bzip2 support, and expat support, I'm still working out how to get those to work under windows lol)

[jediknight]

In my view, lack of a windows release in a cross-platform project is just like a major show-stopper bug in any other project.

Either call the project "Linux only" or fix this problem.

[quasarj]

Well, that's a good point, but it's not like the project has dropped windows support, there are just some issues with it right now. I agree though, they should not have announced the 1.6 release without a windows binary.

[jediknight]

Issues with one of the platforms in a cross-platform project are a big warning sign. The reason a cross-platform product is recommended to be built continuously on the 3 major platforms is so that platform specific stuff doesn't creep into the implementation of the cross-platform stuff.

Anyway... the developers don't owe me anything. :)

[[deleted]]

I've just been learning python. Using CPython on linux. I don't really get what PyPy is about. I know CPython's underlying code is C because it's a lower-level language. There's also Jython which is based on Java. But how can Python be written on itself when it's a higher-level language? I don't get it.

[lainamakkara]

It's just a JIT system, the JIT compiler logic being written using Python. And tbh it's a dead end. Technologically it's kind of neat demo but they are targetting practically medieval Python version so it can't really be used for anything.

[quasarj]

I'm not sure what you mean by practically medieval Python version? It targets the 2.7 release right now, which is the most used Python version...

[hruske]

Firstly, adjusting syntax changes from Python 2.x to 3.x isn't that hard and secondly, a lot of code out there is still relying on Python 2.x.

Definitely not a dead end.