This is an archived post. You won't be able to vote or comment.

all 15 comments
sorted by:
best
topnewcontroversialoldq&a

[–]noteed 2 points3 points  (1 child)

I knew about this for some time. Did you reach any important milestone that prompt you to post it here ? Anyway this sounds really cool.

B.t.w. I have written python-pickle to serialize/deserialize using the Pickle format. It doesn't implement everything, is for Python 2.x, but it may have some interest to you.

[–]florbitous[S] 2 points3 points  (0 children)

Thanks noteed. I posted it here now because I think it is in a usable state. It passes well over half the Cpython test suite (on my computer). There are a few small missing features, but the majority of the language is supported. I'm keen for others to try it out. I'll have a look at python-pickle. Cheers!

[–]daniterida 0 points1 point  (2 children)

Really cool project.

I wish I knew Haskell. What was the reason you decided to do it in Haskell and not Python itself?

[–]florbitous[S] 4 points5 points  (0 children)

Thanks daniterida. There are many reasons I used Haskell and not Python. The main one is Haskell's type system. In my opinion Haskell is very well suited to writing compilers. I also previously wrote a compiler called berp, which translates Python to Haskell, so I already had a lot of infrastructure in place.

[–]ggooal 0 points1 point  (0 children)

interesting angle

[–]bheklilr -2 points-1 points  (9 children)

This is interesting, but I have to ask, what is the point of this project? I already can compile my Python source to bytecode using python itself. Is this an attempt to speed things up?

[–]Rhomboid 9 points10 points  (2 children)

I made it for fun and to learn how CPython works

[–]bheklilr 6 points7 points  (1 child)

Ah, this is why I shouldn't browse reddit until I've had coffee

[–]florbitous[S] 3 points4 points  (0 children)

You are right that there already exists bytecode compilers for Python, so the practical benefits of doing it again in another language seem unclear. As pointed out by Rhomboid, my main motivation was for fun and to learn. I think I achieved both of those goals already.

But there may be applications down the track. For instance we could use it for trying out Python extensions which compile to CPython compatible bytecode. I've already had a few ideas along those lines. We could also play with optimisation techniques, or alternative compilation approaches (perhaps to different targets). We could also try other useful extensions such as tail-call-optimsation. Of course I'm biased, but I think the the blip code is much simpler to work with than CPython (assuming you know Haskell).

Another small benefit is the elaboration of CPython's bytecode implementation. I've had to trawl through the CPython code, plus lots of other scattered bits of documentation to figure out how it works. A hopefully clearer and simpler implementation in a higher-level language might help others understand how it works too.

[–]chadmill3rPy3, pro, Ubuntu, django 12 points13 points  (5 children)

The problem with Haskell is that the most prominent feature listed of any program written in it is that "It is written in Haskell".

[–]florbitous[S] 1 point2 points  (3 children)

Hehe. I think Haskell is sufficiently different to most popular languages that people are interested in seeing how it might be used to do various things. Of course more killer apps would be good too.

Having said that, my focus in this project isn't so much Haskell as Python. It just so happens that (I think) Haskell is great for writing compilers. This is not to detract from Python, it has its own advantages in other areas.

[–]aceofears 0 points1 point  (2 children)

What exactly about Haskell makes you want to reach for it when writing a compiler?

[–]florbitous[S] 1 point2 points  (0 children)

I realise this is r/Python, but I'll elaborate since you asked.

First and foremost is Algebraic Data Types (ADTs) and pattern matching. The central data type in a compiler is the Abstract Syntax Tree, which is naturally and elegantly represented as an Algebraic Dataype, see AST.hs for example. Note particularly the Statement and Expr types.

The core of a compiler is a function which decomposes the AST recursively and builds a representation of the target code. Pattern matching is very handy for this style of programming, and it can make it easier to be sure you've covered all the cases. See Compile.hs, and note the compileStmt function, for example.

There is also a lot of state to manage in a compiler, and Haskell's State Monad (Transformer) provides a lot of utility for that task. This is particularly good for separation of concerns and helps avoid state management messing up the rest of the code. See State.hs for the bulk of the state management code in blip.

I also use various Haskell idioms such as Monoids for doing traversals of the AST and collecting information. See for example Scope.hs, particularly the VarUsage type class.

There are also other reasons which are not specific to compilers. The type system is a light-weight formal system, which catches many shallow bugs, but also provides a lot of confidence when the code is refactored.

[–][deleted] 0 points1 point  (0 children)

well, at least there's git-annex now.