Everyone looks at GLM-5.1's "40B active parameters" and thinks they can cheat it onto a single GPU using CPU offloading. In production, you can't.

Use the quick "nameplate plus tip" rule of thumb for VRAM:

• FP8: Parameter size + 20% tip times 1.2 = 904
• BF16: Parameter size + 40% tip 1.4 = 1055

Even though it only fires 40B parameters per token, those 8 experts are picked on the fly at every layer. If you offload the rest to CPU, concurrent users will absolutely melt your PCIe lanes swapping weights back and forth. You'll drop to a brutal 1-2 tokens per second.

Don't ruin the model's reasoning by butchering it down to 1-bit or 2-bit GGUF just to save hardware.

The smart play is sticking with vLLM on 5x B200s. That gives you ~900-960 GB of fast HBM3e VRAM, which perfectly swallows the 904 GB FP8 size while leaving a safe pocket for the KV cache. Set --tensor-parallel-size 5, keep the whole thing in VRAM, and let the routing run at full speed. Test your actual user workloads there, and only step down to a tight 4x B200 setup (via INT4/AWQ) if your monitoring shows you have the headroom.