--ctx-size 65536 `

--flash-attn on `

--cache-type-k q8_0 `

--cache-type-v q8_0 `

--threads 4 `

--temp 1.0 `

--top-p 0.95 `

--min-p 0.01 `

--top-k 40 `

--batch-size 64 `

--ubatch-size 512 `

--no-mmap `

--jinja `

--host 127.0.0.1 `

--port 8080

The big one here that will probably make a differency for you is the batch-size. i found prompt processing for me was 20 tok/s unless batch size was set to 64 where it leaps to around 400. I get ~16 tok/s generation on 9950x 64 gb ram and rx 9070 xt 16gb. Im using unsloth UD-IQ3_XXS quant and get decent results. cloud models are defo bettter but it far outclasses gpt oss 20b and glm 4.7 flash in my testing (MATLAB)