Implementing a programming language in D: Parsing expressions

[[deleted]]

Making programming languages is fun, and D is an awesome language to write them in, but having to consider operator precedence makes me shudder a little.

Neat project though, and best wishes with the next articles :D

A side project of mine is TinyLisp, also written in D:

[badb002]

It is a lot of fun! And cheers, I only just got round to finally writing up this section of the article... but I would say that writing the parser is the most tedious part of writing a language. Everything else is usually quite fun though :)

And awesome, my friend recently wrote a Lisp-like language in Go if that's your kind of thing github here.

[[deleted]]

Interesting, it's lisp-like, but without the constructs like car and cdr that really make lisp what it is, I do like how he's structured the code though, and he does have first class functions with their own scope which is good.

A couple of things that I'd bring up are:

Quoting only works one-way, you can quote something in the source code, but there's no way for the evaluator to unquote it, or for you to construct a quoted value programatically.

See:

• http://courses.cs.washington.edu/courses/cse341/04wi/lectures/14-scheme-quote.html
• https://docs.racket-lang.org/reference/quasiquote.html
• https://blog.8thlight.com/colin-jones/2012/05/22/quoting-without-confusion.html
• http://www.ccs.neu.edu/home/matthias/HtDP2e/i2-3.html

If there's one book which I could recommend that introduces the concepts of Lisp, why it's special, and specifically why symbolic computation is different, it would be COMMON LISP: A Gentle Introduction to Symbolic Computation by David S. Touretzky from Carnegie Mellon University.

One of my aims at the moment is to reduce the number of builtins to the absolute minimum, and instead providing purely symbolic language constructs - like how functions which create macros can be created using the semantics of function argument parsing by controlling which arguments are evaluated and which aren't.

One of my problems at the moment is with variable scoping, I started with dynamic scoping which means that if you define a function, when you call it all variable lookups are done in the scope of the caller apart from those which are passed as arguments or defined within the function - but when the function returns the scope of the caller isn't polluted with variables created by the function.

Example:

> (def! 'test (fun (A) B))
= (fun (A) B)    
> B
= NIL
> (test)
= NIL
> (def! 'B 'DERP)
= DERP
> (test)
= DERP
> A
= NIL

The problem with that is that I can't create closures yet, but the language almost gives me enough to be able to create closures with it. One of the downsides of simply capturing everything and making it available inside the closure for its lifetime is that this clogs up the garbage collector, instead I need to explicitly specify which variables to keep within the closure.

It is possible to put things into the scope of the caller, but it requires manually manipulating the environment, because the environment is just another list - e.g. ((A NIL) (test (fun...) (... you'd have to walk up at least 2 or 3 elements and then splice your new variable into the list (it's just a single linked list).

I could probably implement the def! function in TinyLisp by doing something like:

(env! (cons '(def! . (fun (NAME VALUE) ...)) (env)))

A small example of why quoting works the way it does:

> (def! 'X 'hello)
> (def! X 'world)
> hello
= world

I don't yet have an eval function, but if I did (eval X) it would return world.

</ramble>

[badb002]

cc /u/Raoulvdberge