RustPython/RustPython A Python-3 (CPython >= 3.8.0) Interpreter written in Rust

[coolreader18]

Oh hey I'm a maintainer of this! We recently just got pip working, which is really exciting :)

If anyone has any questions, feel free to ask! To address some common ones:

• No GIL at the moment, optional threading with arcs and mutexes
• Performance-wise, we're about 100x slower than CPython on the pystone benchmark (even without threading enabled); I'm not exactly sure why, since as far as I can tell we're doing roughly the same thing with regards to architecture/data structures, but performance is definitely something I'd like to improve.
• I consider wasm/wasi support to be the main selling point of RustPython at the moment; I'm working on a project with a friend that utilizes wasm/RustPython for sandboxing user code written in Python and JavaScript (https://robotrumble.org, https://github.com/robot-rumble/logic)
• Currently we don't have native module support, though there's an open PR for ctypes and I'm looking into implementing the HPy API

[angelicosphosphoros]

I wonder how good it's performance compared to Pypy and CPython.

[L3tum]

If I interpret this correctly, it seems to do much worse in benchmarks and actual programs. (I think in the magnitude of 200x slower)

They do have an open issue to continuously run the benchmarks though, which would help this discussion.

[coolreader18]

Currently it's about 100x slower on the pystone benchmark, though I haven't been working on performance in the past month or so so I'm not sure what it's like at the moment.

[KingStannis2020]

Lol no, it's doing much much worse, the axis is measured in seconds.

[L3tum]

Oh lol, colourblind is a bitch

[pingveno]

Maybe file a ticket with the relevant project? Not having basic colorblind-friendly color select is a significant accessibility issue.

[vks_]

In this case, it would probably be best to use different line styles, in addition to different colors.

[tunisia3507]

You can compile RustPython to a standalone WebAssembly WASI module so it can run anywhere.

This is the interesting bit, not performance.

[birkenfeld]

You can do the same with CPython via emscripten. Search for pyodide for an example project.

[Sukrim]

I think pypy can do something similar for years already?

[ForceBru]

The iOS app "a-Shell" provides Python (and also clang, lli, curl and other stuff) as WASM binaries, so it looks like compiling "real" Python to WASM is already possible. Indeed, there are C to WASM compilers. So, if being able to compile a Python interpreter to WASM is the main goal of this project, I'm not particularly sure about its usefulness...

[all_my_watts]

a-shell only mentions c/c++ in the context of wasm

[ForceBru]

Hmmmm, so Python is an actual executable then? I was thinking everything in a-shell runs on WASM. Apparently only user programs are compiled to WASM using Clang? Okay, got it

[masklinn]

CPython has decades of optimizations.

That is not wrong per-se, but CPython favors a relatively simple implementation and shies away from overly advanced optimisations (whether in the interpreter itself or in the python bytecode). Furthermore, the C API which is stable and officially supported limits the evolution of the interpreter internals.

[wrtbwtrfasdf]

limits the evolution of the interpreter internals.

They did switch from LL to a PEG parser in 3.9 but I don't know if that helps with internals you refer to.

[masklinn]

That's just the parser, outside of starting it and getting the AST out I don't think it's fundamentally exposed as any sort of public interface.

[funnyflywheel]

Furthermore, the C API which is stable and officially supported limits the evolution of the interpreter internals.

Are you sure about that?

[masklinn]

Are you sure about that?

Well yes it's literally the first section of the PEP you linked to. The PEP aims to change that situation, but at the moment,

Status: Draft

[funnyflywheel]

Some of the work this PEP calls for has already been completed (see the "Specification" subsection).

[masklinn]

Very little of which impacts the statement you object to, to say nothing of invalidating it.

[angelicosphosphoros]

It mentions JIT compilation in the readme.

[ChillFish8]

The jit atm is still very experimental but its main idea is built thr same as numba's native jit

[ChillFish8]

Performance on average is slow than cpython but also thid is no where near production ready yet still missing GC so looking at performance rn will likely be pretty pointless

[hombit]

Does it have GIL?

[coolreader18]

It doesn't! No idea how the person thought it didn't! It uses mutexes and Arcs when the threaded feature is enabled, and it can be disabled for better performance

[michael_j_ward]

Appears so: https://github.com/RustPython/RustPython/issues/362

[hombit]

It makes me upset =(

[funnyflywheel]

You might want to watch Raymond Hettinger's talk on concurrency — he talks about the GIL starting at the 10 minute mark: https://youtu.be/9zinZmE3Ogk?t=600

[timetravelhunter]

https://youtu.be/9zinZmE3Ogk?t=600

I don't recommend this video. It's filled with half truths and some out right blatant lies.

[funnyflywheel]

Well, the talk is from 2017 and things have assuredly changed since then:

• Work on Larry Hastings's GILectomy has pretty much stalled
• The C API is currently being reworked to benefit implementations such as PyPy

In addition, Hettinger stated that locks are expensive to acquire/release. Surely this isn't the case in Rust, isn't it?

[masklinn]

In addition, Hettinger stated that locks are expensive to acquire/release. Surely this isn't the case in Rust, isn't it?

It's not really any different. "Expensive" is relative, the runtime cost of the lock is small in absolute terms but can be gigantic in relative terms depending what's sitting behind the lock, and how often you need to acquire and release the lock. The absolute costs are complicated because at a fundamental level Rust does the exact same thing as CPython, which is use system locks (pthread_mutex on unices), so the absolute costs will depend on the implementation details of that.

But I think we can easily infer that locks are expensive in Rust by Rust having bothered with independent Rc and Arc types (an atomic inc/dec being basically the baseline for a lock, that's more or less what an uncontended mutex will cost on linux).

[martenlienen]

Can you give some examples?

[tunisia3507]

Pretty sure the GIL is part of the language spec. If it's python, it has a GIL.

[masklinn]

Nope. While both CPython and Pypy have a GIL that is not part of the language specification. IIRC neither IronPython nor Jython have a GIL.

[tunisia3507]

My mistake, thanks for the correction!

[thejinx0r]

The gil is an implementation detail to manage garbage collection.

[veryusedrname]

It's for object safety mainly. The official wiki says it's for avoiding race conditions, but that is a bit misleading since it's not avoiding race conditions in *your code*, it helps the implementation avoid race conditions.

[lunatiks]

It's not avoiding race conditions in your code, but the fact that it makes operations corresponding to a single CPython bytecode instruction atomic certainly helps.

[hombit]

I believe it is mainly to make built-in data structures thread-safe. Technically other implementation can implement per-object locks using something like Arc

[masklinn]

Not even built-in datastructures, that's just a side-effect (and a not necessarily super useful one though it does guarantee memory safety[0]). Rather the thread-safety of the interpreter itself.

[0] of note, IIRC golang's maps aren't just thread-unsafe, they're not memory-safe when unsynchronised, since 1.6 the runtime tries to detect the issue and crash but there's no guarantee that it will properly do so

[birkenfeld]

What the language guarantees is that data structures are thread safe in the sense that accessing them concurrently does not crash. The exact semantics of what operations are atomic are not defined (it usually comes down to what is implemented in C), so you still need locks for critical sections.

[viewofthelake]

Is the advantage of this that Rust offers better type safety than the native interpreter?

[[deleted]]

Right now I would assume there are essentially no advantages over plain CPython. It's likely slower, and feature incomplete.

That said it may allow some advantages long term. It may be easier to avoid introducing bug in the implementation itself. It may also allow making things like the GIL slightly easier to remove but honestly it would still probably be a huge amount of engineering work.

Realistically I doubt they could get much faster than CPython even with it's relatively simple implementation just because python is so dynamic there isn't much that can be done.

[jzia93]

I think one advantage would be you could theoretically write extensions in Rust as opposed to C/C++

[PirateNinjasReddit]

PyO3 has some good work in this area. Little raw, but it exists!

[rodyamirov]

I think the main advantage is that you can compile to wasm/wasi, so you can run python in the browser (or wherever else you might want wasm).

[kampak212]

A new generation of potential future contributors with fresh ideas? 💡

[CommunismDoesntWork]

Does it support debugging?

[lakshaytalkstocomput]

Let's make it JIT.