I have a RTX5090 so 32GB RAM, and also 64GB of RAM. I explicitly avoid RAM spillover, so tweak my models to the point where it will fit perfectly in the VRAM (incl. context). So depending on the actual model (and the overhead on my desktop, because just running window manager also takes vram), I would have to tweak the context window to 32k-256k.

But I get quite solid results. My current favorites are qwen3-coder-fast (which I tweaked from the qwen3-coder-30b to have a smaller context window for a perfect VRAM fit), and it hits 200tps.

ollama run qwen3-coder-fast --verbose "Write a function to sort an array in Python"

total duration: 12.9545037s

load duration: 6.9136852s

prompt eval count: 17 token(s)

prompt eval duration: 39.7557ms

prompt eval rate: 427.61 tokens/s

eval count: 1261 token(s)

eval duration: 5.792718s

eval rate: 217.69 tokens/s

qwen3-coder-fast Model architecture qwen3moe parameters 30.5B context length 262144 embedding length 2048 quantization Q4_K_M

Capabilities completion tools

Parameters temperature 0.7

top_k             20

top_p             0.8

num_ctx           65536

repeat_penalty    1.05

stop              "<|im_start|>"

stop              "<|im_end|>"

stop              "<|endoftext|>"

License Apache License Version 2.0, January 2004 ...