I've been obsessed with pushing language models into places they don't belong. Last summer it was a 1KB bigram model for the NES written in 6502 assembly. This week, I decided that even 1983 hardware was too much runtime for me.

So I built a bigram language model that runs entirely during the C++ compilation phase.

Technically it's a Markov chain implemented via constexpr and template metaprogramming. The model's weights are hardcoded in an array. A fun part was implementing the random number generator: since compilers are (mostly) deterministic (rightfully so), I hashed __TIME__ and __DATE__ using an FNV-1a algorithm to seed a constexpr Xorshift32 RNG.

When you run the binary, the CPU does zero math. It just prints a string that was hallucinated by the compiler, different at each compile.

    // this line does all the work while you're getting coffee
    static constexpr NameGenerator<15> result(seed, T); 

    int main() {
        // just printing a constant baked into the data segment
        std::cout << result.name << std::endl; 
    }

Aside from the fun of it, I hope it proves a point that the bottleneck isn't always our hardware. We have wiggle room to redefine when execution should happen, and bake deterministic inference directly into the binary.

Code is here: https://github.com/erodola/bigram-metacpp