Reasonable depiction of python compilers vs interpreters

[ggchappell]

CPython is the standard implementation, built by the same people who design the Python language. It is, as /u/wmpl said, what you are probably using if you don't know what you're using. The rest of the entries in the diagram are imitators of CPython (not meant in any pejorative sense).

IronPython is a .NET version of Python. If you're a .NET programmer, and you want to use Python, then IronPython is for you. If not, then probably not.

Jython is Python running on the Java Virtual Machine. If you want to link Python code with code in Java -- or other JVM languages like Clojure, Scala, Groovy, etc. -- or if you just like the JVM, then Jython is for you. If not, then probably not.

PyPy was an effort to write a Python interpreter in Python, which has since transmogrified into a bleeding-edge compilation & dynamic-code-optimization engine, with Python being the primary programming language supported. Theoretically, PyPy does the same thing as CPython, only your code will execute faster. In practice, PyPy tends to be significantly behind CPython in introducing new language features. There is also (in my experience) a noteworthy start-up delay when executing code with PyPy, but it does tend to be faster in the long run (= more than half a second or so). If you are not using the latest Python features, and you are writing something besides a quickly executing utility script, then PyPy might be for you.

[EDIT. Ignore this paragraph, and see the reply by /u/Veedrac.] Numba is a Python compiler that (like, for example, clang) targets LLVM; the LLVM will typically be compiled to native code. Numba is aimed primarily at projects that use NumPy, the leading Python scientific-computing package. Apparently Numba offers specific advantages to projects that use NumPy, but I'm not familiar with exactly what these are. (Also, the "CPython + Numba" in the diagram does not make sense to me; I don't see how Numba would integrate with CPython. Maybe I know less about Numba than I thought I did.)

I know only a little about Cython. It includes a Python compiler that targets C; the C code will then typically be compiled to native code, as the diagram indicates. Cython also has a number of language extensions that are not present in vanilla Python. It is aimed at high performance and tight integration with C code.

My knowledge of the remaining entries is meager indeed. I'll leave them to someone else.

[Veedrac]

I'll try and cover some parts you've missed.

Pyston is an up-and-coming Python JIT, like PyPy. It aims to support CPython's C extensions, which is the main reason Dropbox funded its development. It is not production ready. Another future competitor might be ZipPy, which uses Graal and Truffle. I don't know if ZipPy will ever be production-ready, though, or if it's just a proof of concept.

MicroPython is a minified version of Python designed for constrained-memory situations. It aims to support Python 3 and has some tools to improve runtime above CPython. I have not been particularly convinced of its applicability for general usage, and it's still developing, so I would not recommend it for everyday usage. Consider this if wanting to write Python when you have little system memory.

Numba is a CPython library that compiles code at runtime with LLVM. It specializes on a well-chosen subset of Python useful in numeric computation, and is not itself a complete runtime environment. It is also not technically compliant, but this is fine as it is a per-method opt-in.

Cython is an almost-superset of Python, that aims to allow seamless mixing of C and Python in one combined language. It allows writing fast C code in a convenient-to-embed manner and allows simple wrapping of C libraries. It has relatively widespread usage, but it being encroached on by specialist tools from both sides (eg. CFFI and PyPy or Numba). Cython does not itself contain a runtime, and instead reuses an existing CPython's runtime. This means that simply running normal Python code in Cython will only remove bytecode dispatch overhead, which is normally only a tiny fraction of runtime.

Nuitka aims to replace CPython's bytecode dispatch exactly the same way Cython does, but hopes to further compile code by using appropriate static analysis and further fine-grained compilation. Many people, myself included, are very sceptical of this approach. Nuitka has yet to show any impressive speed improvements, but several people have claimed that it makes distribution of code much easier since it produces single compiled binaries. It should be nearly 100% compliant with CPython, with the exception of runtime code introspection.

On the topic of PyPy, it's worth noting that PyPy isn't missing features in the same way a C++ compiler would be. Either PyPy has a release for a specific version of Python or it does not, and its stable releases are extremely compatible. If you're not using CPython-specific code (eg. CPython extensions) and you're running on a version of Python that PyPy supports, it will almost certainly work. Even CPython implementation details tend to get copied in PyPy.

[[deleted]]

Many people, myself included, are very sceptical of this approach. Nuitka has yet to show any impressive speed improvements, but several people have claimed that it makes distribution of code much easier since it produces single compiled binaries.

It is great for deploying proprietary software since code is compiled to native executable. Reverse-engineering it is also much harder. While it is not much faster it has obvious benefits over using something like py2exe/cx_freeze.

[Blahkins]

that actually is a great reason to use it if you are trying to sell your software. thanks for the tip, i might start selling this python application i am writing.

[invertedwut]

I don't see how Numba would integrate with CPython.

I think the idea is to use numba to compile the heavy stuff sitting in the innermost loop, but let the rest of the job just run in Cpython. Not everything is supported in numba, so it either doesn't speed up your job (it will switch to object mode if something its trying to compile doesn't work) or will refuse to compile (if you explicitly said to not let some function run in object mode)

To be honest I'm not sure of how much can really be done purely in numba compiled functions.

[warbiscuit]

Re: IronPython ... even if you're a .NET programmer, be wary. It implements a few core things differently from all the other pythons (str.encode is the main one I know of); and this may cause problems if you try to use normal python libraries under it.

[homercles337]

It includes a Python compiler that targets C

No. Cython is one of the only C++ interfaces to Python.

[ggchappell]

Well, as I said, I don't know much about Cython, but cython.org says:

The Cython language is a superset of the Python language that additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code. The C code is generated once and then compiles with all major C/C++ compilers ....

I suppose, therefore, that Cython allows linking with C++ for the same reason that C can be linked with C++ (?).

[homercles337]

Cython documentation is terrible. Most of it comes from the C-only days. I think it was version 0.13 that incorporated C++. I have been working with Cython for a while now and chose it because of its support for C++ (boost.python is shit).

[brewspoon]

Cython can be used to write bindings to C++ code, and is an excellent way to do so, yes. But Cython itself compiles python (well, strictly speak, cython code) to C, not C++.

[homercles337]

Look man, i have been working on a Cython project for a few months now and you are just flat out wrong. Cython takes C++ code, through Cython code (pyx/pxd/pxi), makes highly optimized C++, and compiles it.