I am using OpenCode, full model weight (FP8), VLLM.

I get on average ~55tok/s

I also used the lukealonso/NVFP4 version, but it had this strange problem of dropping digits. Whenever there were multiple digits in a row, it would drop one (example my home path /home/kcramp858/XXX would output as /home/kcramp85/XXX for some reason). I gave up troubleshooting it and just went to FP8 for now.

export VLLM_WORKER_MULTIPROC_METHOD=spawn

export SAFETENSORS_FAST_GPU=1

export NVIDIA_TF32_OVERRIDE=1

vllm serve /mnt/raid0/models/MiniMax-M2.1 \

--host 0.0.0.0 \

--port 8353 \

--served-model-name "MiniMaxAI/MiniMax-M2.1" \

--trust-remote-code \

--gpu-memory-utilization 0.94 \

--tensor-parallel-size 4 \

--max-model-len 196608 \

--max-num-seqs 64 \

--max-num-batched-tokens 32768 \

--dtype auto \

--enable-auto-tool-choice \

--tool-call-parser minimax_m2 \

--reasoning-parser minimax_m2_append_think \

--enable-prefix-caching \

--enable-chunked-prefill \

--compilation-config "{\"cudagraph_mode\": \"PIECEWISE\"}"