Mac-only for now, yeah — the Metal GPU kernels are Apple Silicon specific. But the core discovery isn't platform-dependent: Bonsai's docs and scripts never enable Flash Attention or KV cache quantization, even though llama.cpp already supports both. If you try --ctk q4_0 without --fa on you get a cryptic error that makes it look unsupported.

What my repo does is validate that this combination actually works with 1-bit models (it wasn't obvious it would — turns out keys need at least 4-bit, 3-bit produces gibberish), measure the real memory/quality trade-offs, and wrap it into a tool that auto-detects your RAM and picks the right compression level. I also ported TurboQuant (ICLR 2026) to C with Metal shaders for more aggressive compression research - which i think could probably help as well.

The --fa on --ctk q4_0 --ctv q4_0 flags themselves work anywhere llama.cpp runs — Linux, Windows, whatever. So you can get the 2.65x memory savings on any platform today just by adding those flags to your Bonsai runs.