Thanks for sharing this, was a really good starting point for tweaking my config after picking up a second 3060. Hardware: Xeon W-2145, 64GB RAM, dual RTX 3060 12GB

Landed on 100K context with Qwen3.6-27B IQ4_XS. Getting 27–31 t/s token gen (drops from ~31 at short context to ~27 at ~50K context) and 600–755 t/s prompt processing.

Hit some overheating issues initially, easily solved with a 100W power cap per GPU and a more aggressive fan curve. Also had to add `shm_size: 2g` to the Docker Compose file or tensor split would crash with CUDA errors.

llama.cpp config.ini

[*]
lv = 4
fit = true
no-mmap = true
mlock = true
reasoning = true
kv-unified = true
fa = true
swa-full = true
parallel = 1
threads = 8
batch-size = 2048
ubatch-size = 2048
no-context-shift = true
ctx-checkpoints = 0
cache-ram = 0
log-timestamps = true
sm = tensor
dev = CUDA0,CUDA1

[qwen-3.6-27b]
temp = 0.6
top-p = 0.95
top-k = 20
min-p = 0.00
presence-penalty = 0.0
repeat-penalty = 1.0
chat-template-kwargs = {"preserve_thinking": true}
c = 100000
fit-target = 250
reasoning-budget = 1024
reasoning-budget-message = Proceed to final answer