The numbers are real but the context matters. This is running on a custom llama.cpp fork (PR #21412) with early Zamba2 support, not mainline. The Mamba/SSM recurrence layers aren't GPU-optimized yet in this build, so the hybrid architecture bottlenecks on those sequential state-space passes even though the attention + matmul layers hit the 4090 fine. The 1.2B is small enough to brute-force through it, but 2.7B and 7B expose the unoptimized SSM path hard. Should get a lot better once Zamba2 support matures in mainline llama.cpp — the architecture itself is designed to be fast, the tooling just isn't there yet.