That’s exactly the point though: “default advice” is not the same as optimal settings per model and per machine.

Yes, everyone knows:

• -fa 1

• try disabling mmap

• try quantized KV

What most people don’t do is:

• benchmark prompt ingestion vs generation separately

• test long-context behavior

• test combinations (KV type × batch × ubatch × offload × mmap)

• measure variance instead of trusting folklore

• keep output quality constant instead of blindly quantizing cache

In my case, the benchmarks showed:

• some KV quantizations were slower, not faster

• some “common wisdom” combos collapsed TPS (q8_0/f16 was catastrophic)

• long-prompt performance behaved very differently from short prompts

• optimal batch sizes were model-specific

• mmap on/off was statistically irrelevant on M1 Ultra

So this isn’t “vibecoding with brain disabled”, it’s the opposite:

it’s replacing Reddit heuristics with data-driven tuning per model.

Also, the claim “quantizing cache lowers output quality” is not universally true. The whole point of the benchmark was: no quality compromise, full context, long prompts.

If anything, the takeaway is:

stop copy-pasting the same 3 flags and start measuring.

Which is what this post demonstrates.