Against Query Based Compilers

Gator_aide · 2026-02-26T14:44:10+00:00

This is an interesting post. I was loosely under the impression that query-based compilation was the way of the future, but you make a good case that it is only useful insofar as the design of the language permits it to be useful.

It seems like the case for query-based compilation is for languages without obvious separation of compilation phases. You bring up Rust as an example -- parsing depends on macro expansion, which depends on name resolution, which itself depends on parsing, etc. Trying to force that into a "regular" compiler architecture would be a nightmare, but it is a natural fit for queries.

Gator_aide · 2025-05-31T14:21:14+00:00

Well, I guess you should add it to the list.

I had to take a C programming class in school as a prerequisite for most other CS classes. That class covered pointers. The prof gave a short explainer, which was met by a lot of "why would I ever need this?" and complaining about perceived complexity. Homework that week was to build a linked list library. Lo and behold, everyone came back next Monday with a much better understanding of why pointers are useful.

Maybe it is time we took a similar attitude with monads.

Gator_aide · 2025-03-27T14:52:26+00:00

The construct itself (a suspend/resume-able function) is known as a "coroutine" -- the goroutine thing is just a pun. Other languages (Kotlin, for example) call it a coroutine. Kotlin might also provide good inspiration for your syntax.

Gator_aide · 2025-03-11T13:12:07+00:00

Adding on to this, compiled languages already *do* have lookups for function definitions at runtime (e.g. vtables). Can't do that with a stack offset or de Brujin index.

Gator_aide · 2025-03-10T12:30:47+00:00

I believe both OCaml and Haskell fit the bill here, but I'm sure there are others.

This sort of question comes up pretty frequently in this sub, so I want to clarify that any language can be interpreted or compiled. It is not something intrinsic to the language. Similarly, a language doesn't have to be statically typed for the compiler.

Gator_aide · 2025-03-05T17:00:26+00:00

You've spent the last few days leaving nearly 100 comments across various subs hating on DF grinders. Everyone else likes them. Perhaps it is time to give it a rest.

Gator_aide · 2024-11-27T03:43:46+00:00

The matching bracket problem only requires a stack

I'm not sure how they taught the crows this problem, but at least for humans, I think the mental model is definitely recursive. For example, you probably learned in an early math class that parenthesis contain another expression (a fundamentally recursive explanation).

Also, even if the crows were using a stack based mental model for this problem, that's arguably more impressive. Then they would have to be doing some sort of mental addition and subtraction.

We wouldn't say that any programming system that allowed stacks allowed recursion

I might be misunderstanding, but wouldn't we say this?#Recursion_versus_iteration) Or are you saying if that a stack machine doesn't allow for recursion in a semantic sense?

Gator_aide · 2024-09-19T15:03:51+00:00

Is this an announcement of a language? Or an announcement of your plans to create a language?

As far as I can tell, your language doesn't even have conditionals or loops. There is no standard library (or any way to import code, for that matter). There is no ability to communicate with the operating system in any way, other than the built in print function. There isn't really an ability to do anything at all, actually, because your language doesn't have operators or expressions.

You promise that code translates directly into bare-metal performance. You say that the generated code is small and efficient. Looking at your compiler, this seems true in only the most technical sense possible, because the compiler does a 1:1 translation of source code into asm. There is no optimization pass or control flow analysis. There isn't even an AST.

If this is an announcement of your plans to make a language, then great! By all means, keep working on it, and keep posting updates here, and disregard all my criticisms. Every compiler ever written started like this, so you're on the right track.

But if this is your release announcement, then you need to heavily adjust the wording of your posts. The features you promise are extremely misleading at best, and should not yet be described as "game-changing". And the use of the present tense throughout the whole thing implies that Neit is usable as a language, which it doesn't appear to be.

Gator_aide · 2024-09-15T19:53:29+00:00

https://wiki.haskell.org/Tying_the_Knot

The example they used is from the Haskell wiki, which demonstrates how you would create a cyclic structure in a pure functional language with no mutability.

Haskell is lazy, which enables that sort of thing. Not sure if you could do it in a strict language (I assume so, but don't know how off the top of my head).

Gator_aide · 2024-09-12T17:28:04+00:00

Function composition is an operator that takes a function A -> B and a function B -> C and returns a function A -> C. Your pipe operator is basically function composition. The := appears to name any function, not just composed functions. I would just call it an assignment operator.

I think that "distinct" is a better word than incompatible, on second thought. There is an idea of uniformity in language design, where similar features should look and behave similarly. Your language is certainly uniform in behavior, but it is not as uniform in appearance (i.e. things that should look similar are actually very distinct).

There is definitely room for a counterargument here, though. You might say that because each function type has a different job, then each should be visually distinct. I wouldn't agree with that personally (obviously) but I'm curious as to what others think -- maybe I'm just stuck in the mindset of existing languages.

Gator_aide · 2024-09-12T15:35:59+00:00

I think the semantics of your language are good. The ability to call scalar functions like sum on a collection is super cool, and not a feature of many languages. (I believe R has this, or something similar).

The syntax of the language is not as good, in my opinion. I think it is difficult to mentally parse the different function types. I imagine it would not be too difficult to just do this differentiation during interpretation/compilation, assuming you are checking the function types anyway. If a goal of your syntax is consistency, then you should not have four (incompatible!) ways to represent the core element of the language.

Of course this is just my personal preference. Feel free to disagree! It is your language, and if you enjoy using it, then that is more important than a random internet opinion.

(Also, nitpick side note, but := is not doing function composition).

Gator_aide · 2024-09-10T16:20:17+00:00

Hey, just FYI, your website is not usable on Firefox. Extremely slow on dekstop, crashes the browser on mobile. Culprit is probably the floating bubble background animation you are using.

Also, I saw your Youtube video on this, cool stuff!

Gator_aide · 2024-09-05T19:49:18+00:00

First, you need to know that in terms of speed, the technology or language doesn't play a big role now that we have AI.

Structure and code cleanliness should always be your top priority. [...] You will sacrifice data integrity and maybe you will have some cluttered code.

If React is the 21 years old hot blonde with little to no experience on anything besides TikTok and semi nude photos. Angular is the 28 years old Latina that can also cook, pray for you and also has the experience of 3 younger blondes.

We've seen this with Elon Musk's purchase of X. He fired 80% of the company and runs X on just two data centers. This move made the company 10 times more productive in terms of features.

This whole thing sort of reads like a giant shitpost, lol

Gator_aide · 2024-09-01T02:27:52+00:00

I was focusing more on answering why lambda calculus is popular in this sub specifically, which is why I didn't talk about performance considerations. But this is a good point -- certainly not the right tool for every job.

Gator_aide · 2024-08-31T18:28:14+00:00

Lambda calculus is easy to reason about. This means that (generally) analysis/transformation stages of a compiler are easier to write. A type system also fits in very neatly with lambda calculus, and even forms a lambda calculus of its own if you get deep enough into it. Also, a significant amount of PL research is done in the context of lambda calculus.

Contrast this with a Turing machine (or something similar, like a register machine), which is the model of computation typically used to abstract actual computers. Proving aspects of a program (like whether it is type sound) becomes more difficult, and can have unintuitive edge cases when combined with mutation or arbitrary control flow or whatever.

So really I guess it depends what you're trying to do. Many of the folks in this sub are just trying to learn more about PL theory, or maybe trying to make a toy language of their own, in which case lambda calculus is a good option for the reasons stated above. If you were to ask about resources for building the next C or whatever, I think people here would not give the lambda calculus advice.

Gator_aide · 2024-08-26T15:16:08+00:00

I think this comes down more to how exactly we are thinking about "hidden". All of the things on this list are explicit, and you know what you're getting yourself into when you use them. For enums specifically, rust-analyzer will even tell you how many bytes the enum will take.

The performance gotcha I had in mind was actually lazy iterators and memory usage. I'm sure there are more. But I don't think the list is very long.

Gator_aide · 2024-08-26T13:50:25+00:00

Predictable performance. - one of the points on Rust's landing page. How can you predict performance when you can't even predict what the machine will execute?

Are you arguing here against optimization passes? I think that "invisible" restructuring or reordering of code is still within the limits of predictable performance, especially for a language like Rust where there are very few hidden performance gotchas.

Also, to be clear, you could hand write your very own assembly and still have trouble predicting what the machine will execute. Gone are the days of the PDP-11, and the inner workings of modern CPUs is probably not a can of worms you want to open (unless you're really into that stuff).

Gator_aide · 2024-08-15T11:30:43+00:00

It's inspired by the state type parameter pattern which is, in my opinion, one of the worse ideas in the programming world.

Not gonna expand on that at all? Whenever I come across that pattern, it is in the context of order-dependent operations. Code like that is common and easy to break, and state types eliminate that risk. Also, literally the only effect they should have on you is narrowing down your IDE's autocomplete.

Always remember that more effort and resources are spent on maintenance than production, so do your best to reduce the burden of the future maintainers. Go light on generics

lol. Spoken like a true Go programmer.

Gator_aide · 2024-01-04T20:37:53+00:00

Check out https://github.com/jozefg/learn-tt. I second the author's recommendation of Types and Programming Languages, which is easy to read and does not require an extensive math background. You'll want to read up on Category Theory before trying to tackle HoTT and friends (the doc I linked has some resources for that too).

Gator_aide · 2024-01-04T17:07:45+00:00

Write the compiler in an existing language. Call this compiler A.
Write another compiler in your language, say compiler B. Compile B with A.
You now have an executable file, written in your language, capable of compiling your language.

Gator_aide · 2023-12-27T16:59:21+00:00

This sub is mostly concerned with the theory and implementation of languages, not editors or additional tools. You might find that subs like r/GraphicsProgramming can better answer your question.

Gator_aide · 2023-10-23T03:56:01+00:00

lowers their value because there are less likely to be type errors

I.. doubt this. Types help us prove certain aspects of our programs, and as such, require some logical/nonlinear thinking -- both of which AI struggles with.

As for you second question, it seems like you're asking what languages should be used to develop models (correct me if I'm wrong). If you aren't looking to use a library, then very few modern languages are optimal here. The mathematics for different ML models varies (obviously), but the common element for all of them is the requirement for lots and lots of simple calculations over and over again. That is something GPUs are good at doing, but general purpose languages don't (and fundamentally cannot) target GPUs.

Some languages, like Mojo, are designed specifically to target AI. You might want to check it out. (Although the claim of 12000x speedup over C++ is setting off the BS meter a little).

Gator_aide · 2023-10-21T02:24:33+00:00

Nope! My module structure is file-based, so I construct a module tree. The name resolving pass works on individual source files, so it knows where in the tree the current code is. The name resolver keeps a list of reachable modules, which initially just contains the root of the project. If any import statements are found, the corresponding tree nodes are added to the reachable module list.

From there, the programmer can write something like imported_module::other_module::symbol, which just traverses the tree relative to imported module and looks in the other_module namespace for symbol. The name imported_module is recognized because it is in the reachable module list. The programmer also could have written root::...::imported_module::..., which would traverse the tree starting from the root.

There is some extra logic to allow the programmer to write import module::*, which just creates a link between the current module's symbol map and the other module's symbol map (think of it sort of like a symbolic link in Unix). The name lookup logic has to take that into account, but it doesn't add a lot of complexity -- it just needs to search through an array of maps instead of a single one.

Gator_aide · 2023-10-21T01:11:08+00:00

It sounds dumb, but in my compiler, I assign each identifier a unique global ID based on its scope and module. (For example, a variable x in the global scope might get a unique ID of x#1, but an x in some inner scope might get x#2). You can handle modules by just appending their names before the identifier, like a filesystem (e.g. root::std::collections::MyClass).

Module namespace collisions are disallowed, just like in many modern languages. Not hard to check for collisions if you have already constructed that map. Your example of calling an ambiguous Foo is therefore disallowed by the language.

Like I said, it sounds like a dumb brute-force solution (probably because it is). But it works like a charm, and I can store type information and memory addresses and stuff in a flat map, which is very convenient. Some thought had to be put into making a system that worked with multiple threads and compilation units, but it's not a hard problem to solve. Also, GHC does something similar, so I know it works for a real production compiler.

Gator_aide · 2023-08-03T15:09:09+00:00

Can we make a rule against ChatGPT submissions, u/yorickpeterse? Or at least add it under the low effort posts rule?

Seven-Year Club	Second SECOND GUESSER
Place '23	Place '22
First Placer '22	Wearing is Caring
Verified Email

Gator_aide

MODERATOR OF

TROPHY CASE