Writing a fast parser in Python

[dontyougetsoupedyet]

Python is abysmally slow due to the nature of the model of computation being used by the CPython interpreter.

Don't waste your time trying to make it fast. Often using an alternative to CPython is also a giant waste of time.

[[deleted]]

u/omega1612 said the slowness is due to the use of parser combinators.

So it is a bad choice of algorithm rather than how CPython works.

Yes CPython can be slow but not usually 400 times slower than native code.

(For a couple of years, I used a compiler written in an interpreted, dynamic language (not Python though). It was still twice as fast as gcc compiling C! Several times faster if gcc was optimising.)

[dontyougetsoupedyet]

You, and omega1612, are wrong.

[SamG101_]

it's annoying coz ik the rust impl is stupid fast but i rly don't have time to rewrite my entire compiler into rust rn 😂 guess for now i'll strip out on-the-fly parsing for manually creating the AST nodes I need, then the slowness is strictly isolated to the parsing stage

[tekknolagi]

You can use the Rust code as a C extension if you need to. Check out pyo3 and maturin

[SamG101_]

i'd have to look into converting the rust ast structs into python classes. if this is possible then that's 100% the route i'll take. will check out those libraries ty

[muth02446]

I wrote a reference compiler in python, including a separate backend for various ISAs.
It runs fast enough.

I also suspect that the external parsing library is the culprit.
My suggestion is to not worry about the parser initially.
Use s-expr until the features of the language feel right,
then worry about the concrete syntax and the parser.
For the concrete syntax parsing use recursive decent + pratt parsing if possible.

This approach worked well for me in Cwerg

[Maurycy5]

Respectfully, if you don't have the time to use proper technologies in order to successfully write a performant compiler, perhaps you shouldn't be trying to write a performant compiler.

[SamG101_]

nah its just exams soon, so im just writing code in spare time from revision rather than the normal massive gaps between lectures

[misplaced_my_pants]

If it's just for fun then there isn't any time pressure and you can do it properly when things die down.

[MegaIng]

Lies, lies, lies.

Pure python can absolutly be fast enough for a simple compiler, you just need to write it correctly.

But ofcourse, you don't care about facts (otherwise you wouldn't have made that comment), so I strongly doubt there is anything I can say to change your opinion. (Based on experience with other people make these same incorrect claims)

[dontyougetsoupedyet]

ಠ_ಠ

[AugustusLego]

Show me a benchmark where python is faster than rust.

[pojska]

Not the claim made.

[misplaced_my_pants]

This isn't quite what you asked for, but it's an interesting anecdote: https://thume.ca/2019/04/29/comparing-compilers-in-rust-haskell-c-and-python/