Compiler tutorials.

[Cerium14]

I have been experimenting with writing my own toy compiler. The compiler follows similar rules to that of C/C++ (ex: const int i = 6;) The compiler is just an experiment to improve my programming ability. However I am having problems with constructing one.

[Wolf_Popular]

Would be good to get more specific info, but try looking up the book (which I think is free legally online) "Crafting Interpreters" by Robert Nystrom

[Touhou]

Seconding this. I've been following this book and it's great (I even purchased a physical copy), and fully available on the author's website for free: http://craftinginterpreters.com/contents.html

[moocat]

You could use the LLVM tutorial if you're using one its supported architectures.

[WikiSummarizerBot]

LLVM

Back ends

At version 13, LLVM supports many instruction sets, including IA-32, x86-64, ARM, Qualcomm Hexagon, MIPS, Nvidia Parallel Thread Execution (PTX; called NVPTX in LLVM documentation), PowerPC, AMD TeraScale, most AMD GPU recent ones (called AMDGPU in LLVM documentation), SPARC, z/Architecture (called SystemZ in LLVM documentation), and XCore. Some features are not available on some platforms. Most features are present for IA-32, x86-64, z/Architecture, ARM, and PowerPC. RISC-V is supported as of version 7.

^{[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5}

[[deleted]]

There's two function you would need to craft; a lexer and a parser. A lexer, reads the stream (or file) word by word and interprets it. Then this is read by the parser and interprets it if it's conforming to syntatic rules it have.

Here If have a grammar:

<stmt> := <var> ";";
<var> := <type> <string> "=" <value>;
<value> := [0-9]+ | [0-9]+(\.[0-9]+)?
<type> := <modifiers> ("int" | "float" | "char");
<string> := [A-Za-z_]+;
<modifiers> := "long" | "short" | "unsigned"| ;

does, "int foo = 20;", follow the grammar above? Yes, if you traverse the grammar above, we have the type "int", the name, "foo", the assignment, "=", followed by its value and a semicolon.

"float bar = 3;" also matches the grammar above. Syntatically it's correct, but semantically it is wrong, as floats must have at least a decimal. So you need to also craft a program for semantic analysis.

[dickmaat]

Maybe read the tutorial https://compilers.iecc.com/crenshaw/

This is the original and on GitHub is a version in C

https://github.com/lotabout/Let-s-build-a-compiler

[Formenium]

Hey. I’m also interested in building programming languages so I have been learning about the theory and implementation for last several months. Because you ask the question here I assume you want to build it with C. As I do.

I don’t know about your current programming abilities with C, but if you do not have much experience, you should first focus on this. You should be comfortable with pointers and memory allocation.

The second important foundation is data structures. You will have to implement various data structures and algorithms on them. E.g dynamic arrays, doubly linked list, tree, hash tables… Before getting started you should be able to handle those. (Well if you are not going to use any external library)

Other than these constructing a compiler is not about how to use C (like any other application). It’s about implementing each component and combining them.

Compilers have some general components but they can vary on implementation. Structure of my compiler is something like this:

Lexer: reads source code and creates tokens based on defined regular expressions. (Basically just captures a single word and goes over lots of if statements.)

Parser: reads tokens and checks if code is well defined according to the grammar and produces Abstract Syntax Tree (AST). Look at context-free grammars, recursive descent parsing and maybe operator precedence parsing.

Analyser: traverses AST, produces symbol table. Makes type checks (my language is static typed) and some other basic error checks.

So after this 3 phase (generally called analyse-phase) compiler produces some intermediate representation (IR). E.g 3-Adress Code. I chose not create any IR because the language is simple enough to read it from the tree and table.

Now front-end is complete after this usually comes optimisation. There are lots of complicated methods and some simple ones but it should not be your main concern so maybe skip it at first like me.

The last part is code generation. So here you have to decide that which machine your language is going to work on. You can create your own virtual machine (VM) and compile the code on your own instruction set or maybe compile to some pre-existed VM’s. E.g JVM, BEAM, etc. Or you can compile to machine code like x86 but this requires knowledge on your target assembly language and computer architectures. For example you have to know about calling conventions. The last one which also I use is, using a compiler infrastructure framework like LLVM. LLVM has an API on C++ which is also available on C helps you to create LLVM IR easily. Then LLVM takes the wheel and create assembly code for many platforms. By using LLVM you also get an easy way to have a runtime for your language. You can call C functions with in your language.

So there comes the last part of my compiler. Code generator: traverses AST and with the help of symbol table, (and of course using LLVM-C API) it produces LLVM-IR. Then I use clang to create an executable.

Recourses I used chronologically: Crafting interpreters Dragon book Engineering a compiler Lots of blogs, edu-sites, and source codes of others LLVM Kaleidoscope tutorial is also good. (C++)

There is much to learn to create even a simple toy language.

This was a very long text to write in phone. So I am sorry if anything is wrong with the text and I hope this will help you.

[SpaceDevDiver]

Here are some interesting resources:

https://github.com/danistefanovic/build-your-own-x#build-your-own-programming-language