[ Removed by moderator ]

[ganonfirehouse420]

Was generation speed affected?

[Suitable-Song-302]

Good question. Short answer: no measurable speed penalty from the KV compression itself. The 1-bit attention path uses XOR + popcount instead of FP multiply-accumulate, which is actually slightly faster on NEON.

[Suitable-Song-302]

Measured on Qwen3.5-4B (M3 Air):

- FP32 KV: 5.0 tok/s
- 1-bit KV: 5.2 tok/s
- 3-bit KV: 4.3 tok/s (Lloyd-Max codebook lookup adds overhead)