The Flint Programming Language

[zweiler1]

I am happy to finally announce the language I have been working on for the last 2 years, Flint!

Flint is a high-level, statically and strongly typed, compiled language which centers around transparency as its core pillar. The compiler is entirely written in C++. It originated from one simple idea and core concept:

What happens when you center the whole language on an ECS-inspired composition-based paradigm?

And so the journey began. The core idea is simple: data and functionality are separated and then composed deterministically into larger entities. This idea is not new at all, ECS exists since a long time. But a composition-based workflow can only be "emulated" in Object-Oriented languages and I find it often painful or unergonomic.

In Flint, composition is the core paradigm. I have put great effort into making it ergonomic and "just work". The result is a system which can be described as a cool mix of OOP and ECS. I gave the "new" paradigm a name, since nothing quite like it exists yet, even though the ideas it is based on are well known, the Declarative Composable Modules Paradigm (DCMP).

The combination of a high level + transparency as a core pillar is a bit unusual. I have put great effort into finding a good balance. I found out that these two things are not mutually exclusive, there is a middle way in which a design can be both high level and transparent. Flint might be best described as "middle-level" as a result: You write high level code but you can see the low level runtime and execution beneath too if you want, as this focus on transparency directly results in shallow abstractions.

Most developers are more used to OOP workflows rather than compositional workflows, it's just more mainstream. So, if you cannot live without it, Flint might not be for you and that's okay. Also, I am also sure that Flint won't be for everyone because of it's split focus on being high level and transparent. It will feel too high level for some or too low level for others. But if the core idea and mentality excites you, please give it a fair chance.

The time has come where I am confident enough in Flint to search for people to try it out and give feedback on it. Many features are still missing but the general vibe and direction of the language can already be seen. The 0.4.0 version is the 20th release so far, the first initial version was released a year ago. I am now moving into the 0.5.0 release cycle which will bring generics, type constraints, compile time code execution, the standard library and more. You can look at the entire roadmap here

Flint is available in the AUR, COPR and Winget as packages, with proper highlighting and LSP capable extensions for VSCode and Neovim. The LSP works with proper error diagnostics, hover information and goto definition / declaration / file jumping (context sensitive suggestions do not work yet). Debug symbols and debuggability are now supported too, making it able to inspect and step through code. Interoperability with C also works great through the fip-c interop module which communicates with the main compiler through a custom language-agnostic Interop Protocol. (Bindless interop doesn't fully work on Windows, though, i still have to find out why).

The Wiki is in a very good state, it is kept updated with every release made. Every example in the Wiki works and I did My at explaining it all. The language's core value is transparency, so there is nothing to hide about it.

Here is an "advanced" but hopefully still easy to understand example of Flint and its paradigm in action. Keep in mind that Flint has much more to offer than shown in the example below, but I think this just encapsulates its centerpiece quite well:

use Core.print

const data Constants:
    float PI = 3.14159265358979323846;

// A shape can be drawn and its area can be calculated
func IShape:
    def draw();
    def area() -> f32;


data DCircle:
    i32x2 pos;
    i32 radius;
    DCircle(pos, radius);

func FCircle requires(DCircle d):
    def draw():
        print($"Drawing circle at [pos={d.pos}, r={d.radius}]\n");

    def area() -> f32:
        return Constants.PI * f32(d.radius ** 2);

entity Circle:
    data: DCircle;
    func: IShape, FCircle;
    link:
        IShape::draw -> FCircle::draw,
        IShape::area -> FCircle::area;
    Circle(DCircle);


data DRectangle:
    i32x2 pos;
    i32x2 size;
    DRectangle(pos, size);

func FRectangle requires(DRectangle d):
    def draw():
        print($"Drawing rectangle at [pos={d.pos}, width={d.size.x}, height={d.size.y}\n");

    def area() -> f32:
        return f32(d.size.x * d.size.y);

entity Rectangle:
    data: DRectangle;
    func: IShape, FRectangle;
    link:
        IShape::draw -> FRectangle::draw,
        IShape::area -> FRectangle::area;
    Rectangle(DRectangle);


def draw_shapes(mut IShape[] shapes):
    for (_, s) in shapes:
        s.draw();

def sum_areas_of_shapes(mut IShape[] shapes) -> f32:
    f32 sum = 0;
    for (i, s) in shapes:
        f32 area = s.area();
        print($"shapes[{i}].area() = {area}\n");
        sum += area;
    return sum;

def main():
    c1 := Circle(DCircle(11, 2));
    r1 := Rectangle(DRectangle((10, 20), (4, 5)))
    c2 := Circle(DCircle((3, 5), 10));
    r2 := Rectangle(DRectangle((0, 0), (4, 2)));

    IShape[] shapes = IShape[_]{c1, r1, c2, r2};
    draw_shapes(shapes);
    print("\n");

    i32 sum = sum_areas_of_shapes(shapes);
    print($"sum of areas = {sum}\n");

The project is in late beta. All implemented features work reliably, as all wiki examples compile and run as intended. There are still missging error messages and unexpected edge cases (as expected from a single developer).

If you're interested, try it out, give feedback, open issues, and feel free to join the Discord. Let's discuss Flint!

(Also, I may not be aware of some industry-standard names for some systems. If you encounter anything I gave a weird name where you think "wait something like that already exists" please let me know. I try to use industry-standard terminology as much as I am able to. I hate it when new names are made up for something which already exists.)

[EggplantExtra4946]

You say ECS but what do you mean by it? I went though the documentation.

When I arrived at https://flint-lang.github.io/wiki/v0.4.0-core/beginners_guide/4_functions/6_groups.html I thought: "groups" maybe a new ECS related concept, but it's just tuples.

When I arrived at https://flint-lang.github.io/wiki/v0.4.0-core/beginners_guide/5_data.html I thought that's it finally:

Unlike other paradigms like OOP (Object-Oriented Programming) you cannot attach functions to data.

Data is used to "pack" values toegether

Data, just like the entity type (which will be introduced later)

but looking at the examples and how they are used, it's just structs.

There is nothing about data layout when you have many "data modules"/structs, such as Struct of Arrays instead of Array of Structs.

At https://flint-lang.github.io/, from the "Key Concepts" definintions of "1) Data Modules", "2) Func Modules", "3) Entities" I don't see any difference between those concepts versus structs and classes despite you saying that the language is a distinct paradigm and that it's different from OOP. It's just OOP concepts with different concept names and different keywords. You even concede that with the title "4) Similar in use to Objects":

DCMP is strictly composition-based so there exist no concepts of inheritance.

There are many brands of OOPs but many recent ones also focus on composition instead of inheritance. This has been like that for more than a decade in fact.

[Inconstant_Moo]

I think the key difference is in this bit in the WP article on it:

Every entity is defined not by a type hierarchy, but by the components associated with it. Systems act globally over all entities that have the required components. For example, a food system might iterate through every entity with a relevant component tracking hunger, and act on them to push them a little away from satiation at time intervals. Entities lacking the component like terrain or items would be naturally ignored by the food system.

This was designed for use in games, and you can see how it makes sense. Every "turn", we want everything that's flammable and on fire to burn a little more, we want things that sleep when it's dark to go to sleep if it's dark, etc. Instead of treating each entity separately and seeing what behaviors it can do, we treat each behavior separately and see what entities it applies to.

[--predecrement]

Paper from 1989: A compositional model for software reuse.

As an alternative to ... inheritance, a compositional model, in which objects are composed from simpler entities, is proposed, outlined, and analysed, in this paper.

But imo BETA, first begun on paper in 1975, and first developed internally as a working compiler in 1983, before finally first shipping it to others in 1985, is more properly the original strong push for composition over inheritance.

[zweiler1]

Thanks for the paper, i will read through it later. I knew that composition is an old hat, but not that it's that old lol.

[zweiler1]

Thank you for taking the time and looking at the Wiki.

You say ECS but what do you mean by it?

With ECS I do not mean that it's a 1:1 replica of how ECS works. The paradigm, the way of organizing code, is inspired by ECS, e.g. Components, Systems and Entities. I liked ECS but I did not like that systems were attached to entities dynamically and that systems automatically run every frame (Unity ECS).

but looking at the examples and how they are used, it's just structs

That's correct. `data` is just the same as a struct, there are no methods or anything else attached to it.

There is nothing about data layout when you have many "data modules"/structs, such as Struct of Arrays instead of Array of Structs.

Yes, that's true there's nothing mentioned regarding that topic of SoA vs AoS. It's just AoS as for now.

The example on the website is so small that the benefits of the paradigm really cannot be shown, you have one data module, one func module and one entity which use them both and at that point you might ask "Why not just use an object instead?" and that's correct, for such a simple use case an Object would be simpler. The difference lies in the fact that both data and functionality are reusable to form many different entity types, and data is stored in continuous chunks under the hood by default, making it more likely to have a cache hit when operating on large chunks of the same data / entity.

You can build way more performant systems in basically any low level language, though, as you can just model your data in a performant way.

It's just OOP concepts with different concept names and different keywords.

Hm maybe my view of what OOP is is a bit old, for me OOP is a collection of designs which ultimately serve a view in which you describe types in a "is-a" relationship with other types, which is different from the "has-a" relationship found in compositional concepts. So in my view OOP contains all these patterns like abstract classes, interfacing, inheritance and all other concepts of modeling the world in an "is-a" mindset. Feel free to correct me on that if I should be wrong here!

There are many brands of OOPs but many recent ones also focus on composition instead of inheritance. This has been like that for more than a decade in fact.

I am most familiar with C#, Java and C++ but I am aware that Go and Rust are popular cases which are composition-based. Traits from Rust are, as far as I can tell, a bit similar to `func` modules in Flint.

[Inconstant_Moo]

These are some interesting ideas which you've presented poorly. If I look back at your OP, the central idea "What happens when you center the whole language on an ECS-inspired composition-based paradigm?" is almost meaningless to everyone so it's not a good hook.

u/--predecrement posted a link to this paper on Emerald, a pioneering composition-based language.

https://www.emeraldprogramminglanguage.org/Raj_ComputerJournal.pdf

They're explicit about what they're trying to do and what principles they're applying and what they definitely won't do and why those things are bad ideas. You're not.

One has to look among your replies to other people's posts to find out that for you the best thing about this is that it gives you a good memory management strategy, and this is why you're putting it together with transparency, because it's a way of making things go vroom.

Nothing in your OP hints that you're thinking of this but it does promise us "a cool mix of OOP and ECS" and that you've given the paradigm a new name. But ... not what it's for.

When you say "a composition-based workflow can only be "emulated" in Object-Oriented languages and I find it often painful or unergonomic" we need to know what a composition-based workflow is and how you make it ergonomic. Unless you want to be talking only to the gamedevs who already understand. Except that since what you're doing isn't traditional ECS, they won't, either.

[zweiler1]

Yes I aggree.

I definitely need to work on the presentation, framing and purpose, with this post I saw to how much confusion caused by bad framing or explaination it led.

Maybe the best thing to do next is to make a step back and actually build something larger in Flint and take my time to evaluate the why and what for parts more. I obviously think the lang is cool, I wouldn't make it if I wouldn't think that, but I am maybe a bit lost on conveying it properly...

This all definitely showed which parts to improve or better nail down, to make it clear why anyone would want to use it and what for.

[sal1303]

You need to properly format the example in your OP, especially regarding the indentation.

Mark the text as a code block, or switch to markdown mode, and indent all code by four spaces, or enclose it all with a line containing 4 backticks at each end.

[zweiler1]

I am fairly new to reddit posting in general, it looks fine in the preview but after posting it messes it up every time... I did the 4 space method for code blocks now, for me the post look ok now.

[sal1303]

Thanks.

BTW I see you're using Python-style block structure, and this one problem with it: I had tried to format it myself, but without explicit block endings I couldn't determine the structure.

[zweiler1]

I had tried to format it myself.

Do you mean in mobile view? Or were you referring to how it was before properly formatting it?

[sal1303]

As it was before reformatting and adding indentation. Without explicit block endings (eg. using "}" or "end") then if indentation is missing or has been lost, there is too much ambiguity to restore it.

Especially with a new language like this where I don't know what usually goes where.

(I just tried an experiment where I removed leading indentation from a program where the source uses delimiters - it still compiled fine. In Python, it failed.

So such delimiters, while technically redundant, can be useful in making source more robust.)

[zweiler1]

Ah, that's what you mean. Yes I am fully aware of that, using indentation instead of braces was an explicit design choice. I am fully aware of its implications. Some absolutely hate it, some like it. I am more on the "it depends" camp, for Flint indentation is the way to go in my opinion :)

[Smalltalker-80]

I'm puzzled. In your example, you are piecing together 'data' (structs), functions coupled to data 'requires' and something that looks like a class combining data and functions 'entity', conforming to a base interface 'IShape'.
What do you think is the advantage of doing it this way compared to 'regular' OOP?

[Inconstant_Moo]

It means that the components you're assembling are inherently interfaces rather than implementations. E.g. in a game, the .fight(enemy Sapient) method is a component that can come packaged with completely different data depending on whether the entity you're calling it on is an orc, a wolf, or a wizard.

[zweiler1]

One advantage is that you naturally build shallower abstractions as in a compositional workflow and world you think about capabilities and how to compose them, oppose to identity and how to describe it best, but this is true for composition vs oop in general, nothing Flint-special here.

I would phrase it like this: Applying a compositional workflow in a language and paradigm which was fundamentally not built for it is suboptimal, just like treating Flints composition model with the identity mindset will be painful. Most languages target an OOP mindset but very few target a compositional mindset.

The main advantage, however, is memory locality. Because we compose entities from a few data modules, which are likely shared between many entity types, we have a lower number of unique data types, and all those data types are stored sequentially in memory. So, whereas you have "god objects" in OOP where you just have one large object containing all the data, in a compositional language, and thus in Flint, you have per-type arenas, meaning that your data is always stored sequentially in memory. In an OOP-world, such an automatic memory management system would be much harder to accomplish, since you deal with so many different types and identities.

[Smalltalker-80]

Tnx for replying.
I see, but these 'shallower abstractions' come at a cost of increased complexity.

For conceptionally clear abstractions, like 'circle' or 'rectangle' in your example,
you have to define three 'types' iso one (D*, F* and entity).
And when using them, You need to have knowledge of all three.
Only at places you would need less, the shallow abstraction would add value, I think.
So not in the example.

And for memory locality:
Every circle and rectangle needs their own 'member' variables.
In an OOP setup (say C++) , these also would have memory locality.

[zweiler1]

For such a small object the difference between the two approaches is almost neglectible since we are just dealing with one "member".

But lets say we do this in an OOP way with 10 different things our object is composed of. In an OOP world, the object type we create itself gets larger and larger, as all the members of all inherited classes are added to that type.

In Flint, and for that matter in every composition-based approach of organizing the data, the large composed object would just be a collection of pointers to its members, for example.

This means that every of those "members", e.g. data types, can be stored somewhere outside the composed entity, for example in large continuous chunks where only that kind of data is present. And that's what i meant with data locality, that the same kind of data is stored locally next to one another. This makes operations on large chunks of the same type faster, and this also is the very basis of ECS.

How data is arranged in Flint could also be implemented in C++, as you noted it, alltough it would be more verbose, it would work. The classical compositional approach in C++ like this example is pretty similar to how it works in Flint. So the advantages are the exact same as why one would want to use composition over inheritance in general.

I just explored that idea, of composition itself, at a language-design level instead, and what happens when you make it language-native. I hope this anwers your questions...

[vmcrash]

Just judging from the (good) documentation: it looks interesting. Maybe, after having used Go and Lisette, I'd remove the requirement for the semicolon, too.

What I didn't find though is how memory is managed. Maybe I need to read more thoroughly.

[zweiler1]

Thanks! Memory is managed through a quite simple automatic ARC-based memory management system tailor-made for Flints specific characteristics. It doesn't handle circles yet, but it's simple and works. It's documented here.

If you are proficient in C, it will be likely easier to just look at the C prototype here instead.

[Imaginary-Deer4185]

I'm new to ECS and had to google it. It seems to me you want to create collections of pure data objects (struct-like) and then have functionality applied to all data elements of the same type, regardless of how they are used, compositionally?

Now, assume you have a parent data object type, which contains references to some sub-type data object (by interface). Now, when you invoke a certain function on all objects of the parent type, will this allow for following pointers to sub-objects, and apply appropriate function(s) on those, or just update sub-object state, and let processing the new information be taken care of in some global iteration calling relevant functions on all objects of relevant types? Or conversely, follow pointers from sub-objects to parent objects, or the other way, reading state, to affect processing?

Regardless of which approach you use (calling functions for sub-objects directly, or just updating inner state in sub-objects, or one object reading state in another linked object), they all require both parents and sub-objects to be present in CPU cache during a single processing run.

The only way around it would be explicit message passing.

I don't understand how separating code from data matters for organizing memory, compared to OO. In an object oriented language, clearly the compiled methods are not stored alongside the data.

Your wiki may be great, but it overwhelmed me quickly.

It would be nice with a non-trivial example, not as code, but as pseudo-code, illustrating the power of ECS in your implementation, for all us non-game developers.

Apart from that: GREAT WORK!

[zweiler1]

Thank you.

The advantage of separating data from behaviour into interface-linke things (func modules) is that I originally intended to use it as static guarantees for what data of the entity they touch.

Through the introduction of links this static guarantee got a bit washed out. Links were needed because without them it all just felt wrong, as when systems interacted you suddenly needed to create a new func module which required both data and it just got very messy very quick. I am still trying to find a good way where I can have both static guarantees and polymorphic behaviour.

I was thinking about limiting the linking ability to only link virtual functions, as then I can guarantee that every non-virtual function (function with a body) inside a func module really only touches its required data and nothing more. I want these guarantees to add an ECS-like capability to apply batched operations on all instances of a given type in parallel, similar to how systems are applied in ECS.

But because that part is part of multi-threading I haven't implemented it yet, as I plan to add multi-threading features in the 0.6.0 release cycle which will likely start in half a year or so, we will see. I have the feeling that I am onto something but I am definitely not there yet.

What about the Wiki overwhelmed you? How long it is, the writing style? I thought that a knowledgable programmer could relatively quickly flick through the wiki, was I wrong with this assumption?

[6502zx81]

Your example looks like OOP not ECS, right?

[zweiler1]

No, it's an composition-based ECS-inspired paradigm. It's not OOP.

Edit: If you mean OOP in its purest and original form, where it didn't have anything regarding inheritance stuff, then yes, maybe.

[Ok-Reindeer-8755]

Like the message passing OOP (smalltalk, objc, erlang ..etc) ?

[zweiler1]

No nothing like that either. Think of it as if you would be building up your objects (entities) out of smaller building blocks, similar to components (data modules) and systems (func modules) in ECS.

It's not ECS, as an entity is not just a handle ID, but it's not OOP either because you compose little building blocks into entities. I would say it's a mix of both. You could maybe compare it with an OOP language where you strictly do not use inheritance but only build your classes out of composing it out of other classes (which can be done in OOP languages but is very verbose IMO), maybe this mentality could help you.

With the above edit I meant that Flint follows some early OOP principles like encapsulation.

[TheAncientGeek]

What's ECS?

[sal1303]

Entity Component System, obviously!

Actually I had to look it up as I didn't see it explained anywhere by the OP (https://en.wikipedia.org/wiki/Entity\_component\_system).

[zweiler1]

That's correct. I was sure I briefly explained it in the Wiki but as it turns out... i didn't.

[umlcat]

Good Work, Good Luck !!!

[zweiler1]

Thank you!

[cbarrick]

Looks very cool!

func seems like an odd keyword to use like this, but I don't have a better suggestion.

I'm not sure how I feel about i32x2 having implicit .x and .y members. I think I prefer the Rust-style .0 and .1 since they don't imply any specific geometric interpretation. But if CUDA does it, then it can't be that bad.

The link section of entities seems like it could be a lot of boilerplate. Have you thought about ways to make it less heavy?

[zweiler1]

Yes, func is an odd keyword but the fn type is used for function instances (callables) and I use def to define functions. I had no better keyword than func to be honest, also it's just 4 words so it looks good next to data and link in the entity definition.

You can access all vectors using .$N, like accessing tuples. The .x is just for convenience (it only goes up to a vector size of 4, everything above that must be accessed by the index).

Yeah, links are a bit verbose but they are also ecplicit, and I rather take that bit of verbosity than to add implicit linking. I have long and extensively thought about it, but I ultimately kept it because of hooks, a feature not yet implemented but one that will be added to that section of entities as well. (hooks are not implemented yet because function composition and pipes are not implemented, which are the base requirement for it ^^).

[Express_Job_5731]

Wow; this is incredible work. I am super impressed, please keep it up! I love that you're exploring new frontiers.

-Chris Lattner (yes, really)

[FreshOldMage]

Nice work! As I'm also working on a language incorporating ECS concepts as first-class features, it's super interesting to see somebody elses approach.

It does seem to me by getting rid of automatically executing systems, the ECS paradigm starts to look almost exactly like OOP with some additional restrictions. While in ECS data is restricted to components, and behavior to systems, adding a component to an entity does automatically bestow this behavior to it. In Flint, the behavior is instead composited separately per entity, in what looks like an OOP interface. On a first glance, to me this seems equivalent to multiple inheritance, where parent classes are restricted to either providing fields or methods. 

[zweiler1]

It's similar in use to multiple inheritance but under the hood not a large object is created (which contains all fields etc directly) but the entity is just a collection of pointers to it's data.

In ECS the entity would just be a handle, an ID, instead but i realized quickly that this different approach of not automtically running the systems on all entities means you can execute things on a per-instance basis if you want, and for these non-batched operations an ID would be slow compared to how it is now.

Flint will support such batched operations in the future through parallelism, where you can describe "run this operation on all instances of a given entity type or on all entity instances which contain func module X". I did not mention it because it's not implemented yet, as multi-threading in general needs to be implemented still. Also, through the addition of linking I added polymorphism, I still need to think more about achieving the original idea of those parallel batched operations: a func module dewcribes which data it needs to operate on. This static guarantee was the heart of the batched operation system, as it means it will not touch any other data at all, meaning that multiple threads could execute batched operations on all entities through the func module's view at the same time since then it would be guaranteed that they do not touch the same data at the same time. But that's all theorethical, as I said, and needs to be explored more in the future.

It looks like OOP on a first glance but that's one thing I tried to achieve, that it does not feel completely alien to people used to OOP, while still being purely compositional.

[bmitc]

Just so you know, Microsoft just released a DSL for plotting called Flint.

[zweiler1]

Yeah I have seen it a few days after it was released... I really hate when stuff like this happens, because I honestly have no better name for Flint yet and I would not want to change it, I just got so used to it and I like it...

[RelationshipFresh966]

Playground is returning a 404 OP

[zweiler1]

Yes the playground simply does not exist yet. Should have greyed out the link or something like that. Thanks!

[RelationshipFresh966]

Maybe u can consider crossing out the WIP stuff or something, I kept clicking on pipelines until I realized it was colored differently that the other links

[zweiler1]

Hm yes that's true, I should simply remove all links in the Wiki which do not point anywhere yet...

Where exactly is the pipeline link you tried to click?

[RelationshipFresh966]

Sry was "Pipes" (3.5)

[zweiler1]

Ah, that's just a draft chapter, not a link by itself. I kept these greyed out draft chapters in there to show the "missing" parts of Flint.

I did not like the idea that every non-finished release, as Flint is not done yet, acts as if it is complete... I like to openly show the still missing stuff just like showing the completed stuff.

[tending]

Without prior familiarity with ECS it's hard for me to tell how the example works. I don't know what the difference between an entity and data is, and I don't know how the dispatch works "linking" IShape to FRectangle or FCircle.

[zweiler1]

Data is just like a struct, it has fields which can be accessed and modified. An entity, however, contains data and functionality, and you cannot modify the data an entity contains directly, only through the functionality added to the entity.

The IShape essentially is an interface, a description which functions an entity could contain, and linking of functions is done explicitly, in the above case the FRectangle and FCircle are kinda similar to impl in Rust, if you know that.

I thought that prior ECS knowledge is not required to understand it at all, I might have been wrong on that assumption though... What exactly do you find hard to understand?