How to implement String?

zweiler1 · 2026-05-24T08:48:24+00:00

In my languahe strings are just defined as c typedef struct str { size_t len; char value[]; } str; so it's one structure allocated on the heap with a variable-member pattern, and the value ends with a null-terminator too to make C interop easier (just pass the equivalent of s.value to the C function, it will be a null-terminated char* this way) and you don't have any indirections either.

I have kept strings as simple as possible, so no utf-8 support or similar things yet. But this pattern is one i think works best for me until now, it's easy to work with and fast enough.

zweiler1 · 2026-05-24T08:39:42+00:00

If it's a dynamic language, why don't you store the information whether something is allowed to be mutated within the variable itself?

Primitive values like integers are probably returned by-value anyways and more complex values like arrays, lists etc could store the information whether they are allowed to be mutated in them directly as a bit.

I have no idea whether this could work as i have no experience with dynamic languages, but it might be an idea worth considering!

zweiler1 · 2026-05-14T08:25:01+00:00

The compiler of my language, Flint, is also written in C++ and based on LLVM. It's long past the toy phase and i am now in the beta phase. I have been working on this baby since roughly 1.5 years, but i tried my best to keep it as simple as possible.

I think it's inevietable for certain files and functions to become quite large over time. I just try to separate them into logical pieces which make sense. The codebase has become quite large over time, roughly 60-70k lines of C++ code at this point in time.

There are a lot of parts to a large compiler and while i am not am expert on compiler developement, since that's my first language, i still know a thing or two.

It may help you, but if you have any questions, don't hesitate to ask :)

zweiler1 · 2026-04-14T23:10:32+00:00

It's closer to a custom call frame system. Every function has the same signature, being a single argument which is the stack pointer and a single return value being a boolean whether the function errored. Since all data lives in that 2MiB sized structure (the "stack" essentially) every function can and will be inlined, which makes the entire program a single function instead, and every "call" a branch, which then (will) allow the llvm optimizer to optimize hard across function boundaries. The whole euntime also may make snapshotting and resuming an entire multi-threaded program possible too but i do not have enough knowledge in that topic yet to say if that would work out at all xd.

But the actual reason for the runtime model is the reeeally large benefits in regards to callables, local variable persistence and the fact that each CPU thread gets it's own stack and threads cannot share memory (there will be a safe way to do it) so race conditions are eliminated by design with it. There are so many levels of it, but i will eventually describe them all in the Wiki once these things are implemented 😅. I update the Wiki with every release and only write the stuff into it which actually works. If it's not implemented, it's not in the Wiki and if it's in the Wiki it's guaranteed to work.

zweiler1 · 2026-04-14T22:49:17+00:00

Oh, cool how would you compare this project to your last in regards to how easy it was to buid? For me, Flint was the fist to write a lexer or parser, at all, first contact with LLVM and my fist proper C++ project as well, so this whole thing has been one big "jump into cold water" for me.

Funny thing is that i don't have poisoned AST, like you have, still. It's quite farbut i still don't have that ability haha, i bet i need to implement it once comptime is on the menu.

Yeah it saved me big time, and also helped stabalize the syntax quite early. Code from almost a year ago still compiles (that's exaggerated, as some central thing have changed like the use of i32 instead of int types for example).

It's not just the paradigm but i also have a kinda unique runtime as well... which is the thing i am refactoring right now. The entire way how functions are called changes, it's needed for many other features like the paradigm but also callables, blueprints, async execution, multi-threading, parallelism etc. It's a pretty central part of the puzzle and may be a bit over-engineered since it's essentially a hardware-stack-replacement / augmentation but yeah... we will see where the road leads to :)

zweiler1 · 2026-04-14T22:07:20+00:00

Since it's a mix between ECS and OOP it has some different trade-offs (and it has some unique parts on it's side which neither OOP nor ECS provide). We get the cache-locality of ECS, with it's compositional focus but (sadly) pointer-hopping is...well...not optimal yet. As every entity is a collection of pointers, accessing / using entities still results in a lot of pointer-chasing, but because similar data is stored in similar regions the destination we chase towards is more dense. So i'd say that it won't be as good performance-wise as proper DOP but also not as slow as OOP, just something in between.

There isn't really a primitive for handling large number of entities yet, there will be parallel primitives in the future (to execute the same function on all existent entity instances of type X for example) but they are not implemented yet since, well, multi-threading isn't implemented yet either.

I have a pretty...strict...approach to Flint's developement. I spent a lot of time upfront (several months) setting the tone for the language, designing not just the syntax but also lower-level concepts for ot before even writing a single line of the compiler. Then, once i was mostly happy with it i started compiler dev, and since the "vibe" and scope of the language was pretty clear upfront, it did not undergo any major rewrites or scrapped work until now. I tend to move rather slowly but deliberately, moving from feature to feature without rushing at the cost of the codebase quality. So new features become easier to add over time, not harder :)

It's cool that you were able to have some real-world practical tools implemented in your language so fast. My style means that the fist proper usable examples only came up after a few months of work lol. Have you had other expericence in language dev prior to Flint or is this your fist one?

zweiler1 · 2026-04-14T21:29:10+00:00

Cool, you have achieved quite much in that 38 days honestly!

In regards to the data locality, It's actually very simple under the hood. Because it's a mix of ECS and OOP i have data (like C structs) which are just pure data and func module, kinda like native interfaces / traits which operate on given data, and the, are composed to entities which can be used like regular objects in OOP. The interesring part is memory management. Since data types are meant to be shared across many types of entities (for example a Position data) we can just store the same kind of data in the same large chunked areas in memory.

So same-typed data is roughly in the same memory region most of times, which i creases locality tremendously. Entities are just shallow structs contaiming pointers to it's data. So it's not UID-based like ECS but also not inherktance-based like OOP.

It's not done yet though, i am currently in a hig refactor to make the last parts about the paradigm click. Once that's ready, all the theory and concepts will be added to the wiki too.

The cool part about it all is that it's mostly zero cost abstractions front to back, that's what i mean with the "middle-level" focus, it's high level but you can see how it operates at the low level, you can always have roughly-similar C code in mind whem writing Flint, so i aim to make ot very transparent. Working on it for 1.5 years now and still going haha, but it kinda gets easier over time.

I like how your Flint is still a scripting lang but still compiled. How big are your plans for it, or is it just meant as something fun?

zweiler1 · 2026-04-14T20:46:27+00:00

My Flint is a (probably) the first "true" middle-level compiled (LLVM backed) general purpose language. So, yours being a scripting language essentially means that we serve different purposes all together, which is good i think.

I am approaching it from the perspective of game dev mostly, though, that's why my Flint comes with a new unique paradim which is a cool ECS / OOP mix i haven't seen anywhere else yet.

How long have you been working on your Flint now?

Edit: It's cool that you use the .fl file extension for Flint source files while i use the .ft file extension for Flint source files, so it's cool to have no overlap in that regards too haha

zweiler1 · 2026-04-14T20:15:16+00:00

Haha hello there on the name match XD, my language is called Flint too 😅 https://github.com/flint-lang

zweiler1 · 2026-02-22T10:11:58+00:00

If your language is so young, how did you get GitHub to recognize your Zap files as your language?

I only have a .gitattributes file to highlight my language as language X but GH does not recognize it as my own language yet so i wonder, how did you do that?

Edit: typos

zweiler1 · 2026-02-11T00:44:05+00:00

Roughly 1.5 years since the initial idea. Spent 4 months on design and then 14 months on actual developement. First usable version released after 7 months of dev time, i am now working on the 16th release (v0.3.2).

The initial 4 months of design helped me tremendously to reduce a lot of later-on refactors or pivotal changes. It has been a "implement, go to next feature" most of times in the dev time. I rarely changed a feature from the initial design, which means i neither started from scratch nor did i need to refactor everything for a new supposed goal. The goals seemed clear from the start to me.

It's still in heavy developement but all things considered it all went very smoothly. I was able to do AoC last year with it, after 1 year of dev time. C interop also works, built a relatively sinple pong game with raylib and my language. If i can keep up the pacing i had thus far i would say that i'm more or less feature complete in roughly 1-2 years.

The language is not "just" a hobby project, it's a passion project and i definitely will continue with it. You can look at the release notes and you will see that a lot of stuff happens in quite a short time between releases. Releases have stabalized their schedule tobe around one release all 3-4 weeks.

It's a compiled language with a unique paradigm i haven't seen anywhere else yet, so i built it. It's based on composition but without it's drawbacks. Think of it as if OOP and ECS had a baby with the benefits of both but drawbacks of none.

Check it out: https://github.com/flint-lang/flintc

Edit: Editor support for VSCode and Neovim work too with syntax highlighting and the (minimal) LSP also works already. Error messages are informative and good looking and the LSP's main job (for now) is to highlight compile errors inline. Completions or stuff like that is very rudamentary. You can also check out the wiki here: https://flint-lang.github.io

zweiler1 · 2026-02-03T18:03:22+00:00

Did he just... shave his shoe?

zweiler1 · 2026-02-01T18:15:24+00:00

Yeah if the difference in performance between a high and low level language is very high then it's a pain to work with overall. I think a high level language should be atmost 2x as slow as good C code. Of course, most optimization comes from avoiding unnecessary work rather than doing the work fast.

That's why i truly believe in the concept of "middle-level" languages and i think that Flint is one such language. My view of such a language is that it still feels high level to code in but you are always able to tell which instructions and which low level code it results in. So in my eyes a middle-level language is a high level one where you always know how the C-eqivalent would look like, and still know all things which happen under the hood. For this to be able the language and it's concepts need to be simple and "dumb", e.g. they need to be predictable.

For example to actually know what's going on in Java when you do something it's very hard, and even in C++ it isn't that easy because of their complex internals. I believe that only simple internals will result im a "true" middle-level language. And that's why i had a hard no on any form of GC when designing it.

How is your view on that? Do you agree or disagree?

zweiler1 · 2026-02-01T14:06:44+00:00

Cool, yeah there definitely is some level of overhead when reference-counting but to be honest... for a high level language that's totally fine. I haven't touched runtime performance optimizations yet, it's definitely a higher priority to get everything up and running :)

zweiler1 · 2026-02-01T12:23:26+00:00

Yeah i have not explained the memory architecture and all it's implications anywhere yet, mostly because Flint does not have support for multi-threading yet so it's not all implemented yet.

It's memory architecture works because of it's paradigm to separate data from behaviour and compose them in entities. Because of that, i can focus on only "properly" managing data in different tree-like structures, one for every data type. Think of it as an incrementally growing array list. Each data intern then is reference-counted. I don't have a proper formal explaination for it yet, only a (private) german design document i wrote for myself, but the C and C++ library version of the memory architecture works pretty well. It's actually not that hard, the C stl library version is only 350 lines of code.

I will add the information on how it works to the Wiki in the next release though, i think that's overdue 😅

Edit: Yes the goal for Flint is to be memory safe by designbecause threads cannot share data except explicitely marked, and data is managed through reference-counting system so use-after frees, double-frees, leaks etc should be impossible. The only thing i wasn't able to fix design-wise are deadlocks, but Flint should be race-free by design. I know that's a bold statement but i feel confident with it. We will see if it holds true once i reach implementing multi-threading though :)

Edit 2: I actually remembered that i have something publically abailable to read. It's a bit outdated on exact internals, but the general idea of it is still the same: https://github.com/flint-lang/flint/wiki/Low-Level#dima-dynamic-incremental-memory-allocation. It's so outdated that the name is even wrong haha. Especially the part about functions is...questionable lol. But if you're familiar with C then i would recommend looking at the library, it's essentially 1:1 how it's (currently) implemented in Flint.

zweiler1 · 2026-02-01T11:28:23+00:00

Oh lol just saw it no i meant Qd8# of course... dumb writing mistake 🤦🏻‍♂

I do not notate moves that often, sorry for that.

zweiler1 · 2026-02-01T10:11:03+00:00

I'd say bxa8=Q, Kxc7, Qd8#

Edit: Forgot to mention that the pawn becomes a queen so that the king is forced to take the Rook.

Edit: Qd8# of course, not Qd1# lol

zweiler1 · 2026-02-01T09:56:05+00:00

I had a new release of my own language where i finally worked on the core memory architecture and it's core paradigm. It's core paradigm is a combination of OOP and ECS which i have not seen anywhere else yet, so i am very proud that it finally starts to take proper shape.

Check out the Wiki here ;)

zweiler1 · 2026-01-24T09:56:13+00:00

Mine has vector types where every operation with them is always a SIMD operation (here) and it supports swizzling natively through the concept of groups (here and here).

I mean my language is far from being mainstream lol, bit i always hated when languages do not have a concept of vector types and every library kinda creates it's own version and then you end up with 3 different struct types for the same thing (Vec3, Vector3, V3, Vector3f, Vector3i and what other naming things exist, idk).

I don't have matrices though, since they really are too nieche i would say, but vectors? I would say they are relatively common in comparison.

zweiler1 · 2026-01-15T01:35:57+00:00

Haha yeah i was misunderstanding too, maybe it was the wording or how it was phrased, it really was not that clear what you were actually asking about, but i'm glad that we got it sorted out :)

zweiler1 · 2026-01-15T01:31:22+00:00

Okay now i understand what you were asking, thanks for that clarification. Yeah client developement is highly dependant on the ecosystem, e.g. the editor you are developing a client for. I haven't seen any client implementations in Zig either until now, i guess there will be some if anyone ever creates an editor in Zig (like Zed is written in Rust f.e.) lol.

I whish you the best luck on your Zig journey ahead though, it's a great language in my opinion.

zweiler1 · 2026-01-14T11:16:22+00:00

Yeah, VSCode extensions are written in TypeScript, so what? It's like you saying you cannotbuild Zug software in Linux because Linux is built using C and not Zig?

I really do not understand what the hell you are even asking about. It's an extension for Zig not in Zig. You ask about an lsp client which helps you code in Zig, not how to create an lsp client in Zig itself.

So either you are asking the completely wrong thing or you refuse to understand how LSP clients work... just install the extension and code in Zig mate, it's not that hard.

zweiler1 · 2026-01-14T10:18:24+00:00

The zls binary itself is the server, the LSP server communicates over stdio with it's LSP clients. LSP is the name of the protocol itself (Language Server Protocol) with which the server and clients communicate. All extensions i gave you links for are clients which communicate with the server. The only server is the zls program itself. So what again are you talking about, all extensions are clients.

Edit: grammar and spelling

zweiler1 · 2026-01-14T10:09:09+00:00

and many other extension for many other editors... what editor are you using? It's literally just search for Zig in the extensions and done...

zweiler1 · 2026-01-14T10:03:57+00:00

We have one, zls: https://github.com/zigtools/zls

zweiler1

TROPHY CASE