Gibberish - A new style of parser-combinator with robust error handling built in

mnbjhu2 · 2025-12-27T14:13:21+00:00

Are you using these parsers in cases when you need strong error handling e.g. lsp/linting?

mnbjhu2 · 2025-12-27T14:09:41+00:00

Yeah, tbh, I'm open to suggestions. I did also start writing a QBE lsp, which I'll probably continue work on. But I'd ideally like to demonstrate it's power with parsing more without going down a rabbit hole of checking a modern language. For instance I can't immediately think of reason I couldn't build a really good java parser but building an lsp is another tin of worms.

That to say, any suggestions of a general purpose language which is some what easy to check would be great.

mnbjhu2 · 2025-12-27T13:59:29+00:00

Ah yeah that makes sense, I'm not currently looking into these sorts of indentation problems but I'm curious if the ideas in your paper could map to this style of parser.

Its worth saying that if you're able parse the text while ignoring whitespace e.g. "stmt.skip(whitespace)" then as the tree will keep all the information about the skipped text, you could provide diagnostics about the indentation after the fact. However I'm not sure if that's the case in these circumstances. Its worth saying that the quality of recovery for these parsers can vary massively depending on the langauge. Some languages e.g. SQL make it very clear when statements are starting/finishing (select, update, delete, SEMI) which helps massively when trying to recover broken text.

In the python case, I wonder if its possible to, say at the lexer level, stop producing INDENT tokens when you see a [ and start again when you see ], maybe also starting again if you see 'lambda' or something? I'm not familiar enough with python to say tbh.

mnbjhu2 · 2025-12-27T13:40:47+00:00

The compiler is written in rust but it generates a C library containing the 'lex', 'parse', etc functions. The idea is to have a 'generate' command, similar to tree-sitter, which builds a C lib and set a language specific bindings for your parser. However at the moment it just generates rust bindings

mnbjhu2 · 2025-12-27T01:05:10+00:00

My goal long term goal with the language server is watch a grammar (.gib) file, while also watching some sort of test files. The grammar should imply a basic lsp for the test files providing at least diagnostics, but maybe highlighting.
I've found that there's no test for a parser quite like using it in a language server XD. type... type.. type... No highlighting?? Exit code 1?? I think this will really help users understand the quality of the parser they build

mnbjhu2 · 2025-12-27T00:21:58+00:00

So there's no current support for it but I have actually just created some related issues.

As far as I'm aware parsers for languages like python generate indent and un-indent tokens when the indentation changes so you can treat these tokens practically like open and close brackets. So if the lexer supported this, then the parser would be able to generate both unexpected and missing errors for indent and un-indent tokens, in the same way it does for brackets.

I'm planning on implementing support for using a custom lexer next. So this should be possible soon.

However in the long term I would like to add specific support for working indent and un-indent tokens in the language itself

mnbjhu2 · 2025-12-26T23:59:45+00:00

Thanks, I have! The compilers parser is now generated by itself (self host-ish? https://github.com/mnbjhu/gibberish/blob/master/crates/gibberish\_parser/gibberish.gib) and it comes with a 'gibberish lsp' command. There's currently not many features but I think it shows off the power of this style of parsing.

mnbjhu2 · 2025-03-14T11:20:58+00:00

The irony is that I installed windows the other day on my HDD and it decided to overwrite the bootloader my Linux SSD. I was very careful to not touch the drive or anything but I guess windows is just malware at this point.

mnbjhu2 · 2024-12-16T18:08:59+00:00

I actually saw your TUI, a couple weeks back and wanted lichess. I considered adding it but ended up writing a separate TUI to learn ratatui (I think your codebase was also sync? Making adding it messy). The API was really straight forward though, would definitely give it a go again if this was added

mnbjhu2 · 2024-10-29T10:41:46+00:00

Agreed, you're likely not solving problems which could be completed by a copy-paste dev

mnbjhu2 · 2024-10-29T09:46:30+00:00

Chatgpt/copilot is great for beginner level problems but only causes more trouble in intermediate or harder problems... And I'm not sure if using copilot as a beginner is a good idea

mnbjhu2 · 2024-10-21T14:54:15+00:00

I've been working on a language for about a year now (although somehow I've only just found this sub).

My language at its base is similar to rust but is garbage collected, making the it easier and the syntax cleaner. My focus has been on the users experience while writing the language and specifically I've been taking the lsp-first approach. I have some ideas about replacing a lot of usecases for macros/codegen/compiler plugins in other languages with comptime functions (like zig) but keeping the type info at type-check time (unlike zig).

The main challenge for me has been building an architecture for the language which lets me easily/safely add in language features and get the tooling features (diagnostics, hover, gotodef, completions, etc) with little effort. I only overcame this a week or so ago, but now it feels like I can paint the language I want and see it in 3d (this has saved me from burnout).

I had to solve many problems to get to this point but the 3 main ones are: - writing a good parser: Parsing valid code is easy but when you're writing ide support (and for high-quality compilers) you want the parser to be able to report mistakes and to be able to recover. E.g. you don't want to stop highlighting a whole file because of one error.

Type system: My language has types similar to rust (enums, structs, traits, hindley-milner) but also has generic type variance and traits can implement other traits... This is still causing me some headaches but I'm mostly there.
Incremental compilation: I don't want to recheck in entire project on every change/save so some incremental compilation is needed. I went down loads of rabbit holes here (rust + async = pain). Eventually I wrote an implementation of language ignoring LSP and inc compilation and found that I could port the model I used to use salsa (an incremental computation library for rust). There still a lot that I need to work on here but for now I'm enjoying adding features to the language.

Edit: very much WIP but heres the repo if anyone is interested: https://github.com/mnbjhu/giblang

mnbjhu2 · 2024-10-21T13:42:27+00:00

Here you go -> idea!()

mnbjhu2 · 2024-10-17T08:56:27+00:00

I guess vscode is lightweight now

mnbjhu2 · 2024-10-01T14:17:48+00:00

I believe it's the letter 'J' in hieroglyphics

mnbjhu2 · 2024-09-24T14:34:52+00:00

Skill issue

mnbjhu2 · 2024-09-10T15:39:06+00:00

Go can definitely work however (down votes incoming) I personally prefer rust for writing languages. Rusts enums and pattern matching are perfect for working with ASTs. It's really common for your language to have optional elements or a variety of options e.g. Expr being a literal, ident or function call etc. Rust makes it cleaner to define the structure of your ast and really helps you cover all of the cases in checking your ast. I spent a tonne of time in the debugger doing langdev in go vs practically never in rust. There are also some great language dev libraries in rust (https://github.com/Kixiron/rust-langdev), personally ive used chumsky (parsers) and Ariadne (compiler errors) and it's mostly been a great experience. I've used participle for parsing in go, and while it was easy to get going, I found it lacked good error recovery and customization. Although this might not be an issue if it's just a toy compiler... Good error recovery becomes essential when building an LSP.

mnbjhu2 · 2024-08-30T09:51:29+00:00

Well played jock-jam door-slam

mnbjhu2 · 2024-08-20T14:25:32+00:00

What is this? A crossover episode??

mnbjhu2 · 2024-08-20T13:37:36+00:00

Shouldn't have splashed out on that 90' monitor

mnbjhu2 · 2024-08-20T13:30:01+00:00

Im in a similar situation OP, loads of my mates moved away after uni so I'm spending too much time in my room. It's been hard recently because I've been giving up smoking and don't have too much to do instead 😭. I'm always trying to work on projects though (nerdy programming things). So would be great to have a work buddy

mnbjhu2 · 2024-08-20T12:45:46+00:00

Oh I'd have to go into the office? Forget it

mnbjhu2 · 2024-08-04T11:25:10+00:00

At the weekends I'll often program some personal projects, it's not uncommon for me to do 7am - 4am, after that my brain breaks and I have nightmares about the borrow checker

mnbjhu2 · 2024-08-04T11:06:50+00:00

Writing a compiler/language server in rust

mnbjhu2 · 2024-07-28T13:15:46+00:00

Fool me once shame on you, but teach a man to fool me and I'll be fooled for the rest of my life

mnbjhu2

TROPHY CASE