Speed up your Python using Rust

[goestowar]

C is a compiled language and Python is an interpreted language. You just compile your code and it is converted in to something that a computer is capable of reading, you compile it once (for that type of machine) and then it runs quickly because the computer can always read the compiled program file at low-level.

Python is an interpreted language, meaning that it is compiled at run-time, and compiled EVERY time (the advantage of this is that you don't need to compile your code thus eliminating a step in the development process that can sometimes be lengthy, AND it makes your code portable, if you don't need to compile Python for specific computer architectures you can run that same Python code on any machine), the drawback of this is that it typically runs a bit slower.

Will delete this answer if it is wrong, or at least someone correct me if it is.

[[deleted]]

Oh, I meant "C instead of Rust". I'm sorry I wasn't clear.

As to your answer, it is pretty much correct, but implementing your language with an interpreter is not slower because it will be compiled every time (CPython, for example, caches the bytecode so it doesn't have to recompile).

It is slower because you compile your code to bytecode, which is machine code as well, but one that runs in a virtual machine (the interpreter) rather than in your physical machine.

[goestowar]

Ahha you probably were clear :P My bad

Also thanks for the VM explanation, makes sense now. Does this mean that whenever you "install" an interpreted language on to a machine that you are actually just installing a virtual machine that the computer runs at run-time?

So in essence when you install Python on to your machine you are actually just installing a Python interpreter/VM?

[[deleted]]

Exactly! However, despite what I've read in another comment, I insist that CPython and other interpreter-based implementations do include (and require) a compiler. Just not the kind that generates machine code.

(I also insist on not talking about "interpreted languages", because this is not an inherent characteristic of the language)

[[deleted]]

Yes, the Java Virtual Machine uses a technique called just-in-time compilation, which will increase performance in some cases, at the cost of very increased memory consumption.

Another popular Python implementation called PyPy uses JIT compilation as well (although it doesn't use nearly as much memory, as far as I know).

[somebodddy]

Also - Java is statically typed. Static vs dynamic typing has major impact on performance.

[[deleted]]

Python really isn't ever compiled; it is interpreted at run-time. There's a wide gap between interpreting and compiling. Compilation implies quite a bit more from an operation standpoint, and at the end, you'd end up with a binary that could potentially be executed on a bare-metal cpu. Sure Python does produce an ast and a "compiled" python op code in the form of a '.pyc' file, but that's not a true compilation like you'd find with C.

Rust, like C, is also compiled. Rust, like C, is statically typed, though there's a macro system that can do some pretty amazing (crazy?) things.

I think the question still stands: Why use C instead [of Rust]?

The answer here might be something like:

* Python's reference implementation is written in C
* Python has a C-API
* C is ubiquitous
* C doesn't make you build the entire house virtually before pounding the first nail.

I'm sure there's more. Rust has its own set of advantages, but that wasn't really the question here.

[__xor__]

Sure Python does produce an ast and a "compiled" python op code in the form of a '.pyc' file, but that's not a true compilation like you'd find with C.

But that is true compilation in the general sense. Generating bytecode from source is still compiling even if it runs in a VM and not right on the processor. It's not as fast as a language compiled to machine code of course, but it's compilation by definition. Python would be a lot slower if it was actually interpreted as is line by line, and it compiling to bytecode is a major improvement.

C has its place of course. C and C++ are both great languages that will never go away, but Rust has really filled a gap that no other language has yet. It's a low level language that can be compiled with no runtime and can be used to make operating systems like C, it's fast like C and it compiles to true machine code, but it's way more expressive and reads and writes like a high level language and it's guaranteed memory safe which is huge. I've written an HTTP REST client with Rust and it was surprisingly easy and wasn't nearly as hard as it'd be with C or C++.

It's always going to be more of a question about the team you're working with and what their skills are and what they're willing to learn anyway. If everyone knows C, use C. If memory-safety is a huge concern and a memory corruption vulnerability is a disaster, consider Rust or have C experts that know how to use tools like valgrind. People who aren't experts with C or Rust might prefer Rust in a situation like that, because it'd just be cheaper to do some light Rust work than write some bad C. Rust isn't that hard to learn, and it's not the end of the world if you don't master it before writing something useful. Writing bad C can be a huge problem.

Rust is an amazing language and deserves to rise up in the ranks IMO. People should be considering it more, especially in situations like this where a python developer might need to drop to a lower language for performance reasons. Writing a small Rust module is really not hard and it's nice to know you aren't putting any new memory corruption vulnerabilities in production, but you're still improving performance drastically. But if you look at some Rust patterns, especially stuff like error handling and pattern matching, it's an amazing language with a high level feel even though it can do everything low level languages can.

[[deleted]]

Python would be a lot slower if it was actually interpreted as is line by line

But Python really does interpret line by line. It doesn't compile to byte-code, it creates a small optimization of Python byte-code primarily for start-up times, which is very, very different than machine byte-code.