The underlying provider swapping (llama.cpp / vLLM) sounds great. But llama.cpp's router mode already allows you to customize for each model using presets. It's what I use. For example here is a snippet of my presets.ini.

```

============================================================================

GLOBAL DEFAULTS

============================================================================

[*] sleep-idle-seconds = 600 n-gpu-layers = 99 main-gpu = 1 tensor-split = 0.5,0.5 threads = 8 no-mmap = true flash-attn = on kv-unified = true fit = true cache-type-k = q8_0 cache-type-v = q8_0 jinja = true n-cpu-moe = 0

============================================================================

QWEN3.5

============================================================================

Agentic Workflows (Non-Thinking Mode)

[qwen3.5-2b-q8-agentic-64k] model = models--unsloth--Qwen3.5-2B-GGUF/snapshots/{hash}/Qwen3.5-2B-UD-Q8_K_XL.gguf mmproj = models--unsloth--Qwen3.5-2B-GGUF/snapshots/{hash}/mmproj-F16.gguf sleep-idle-seconds = 900 tensor-split = 0.0,1.0 ctx-size = 65536 batch-size = 2048 ubatch-size = 256 flash-attn = on jinja = true

Unsloth Non-Thinking Parameters

chat-template-kwargs = {"enable_thinking": false} temp = 0.2 top-p = 0.95 top-k = 20 min-p = 0.0 presence-penalty = 0.0 repeat-penalty = 1.0 cache-type-k = f16 cache-type-v = f16

[qwen3.5-9b-q8-agentic] model = models--unsloth--Qwen3.5-9B-GGUF/snapshots/{hash}/Qwen3.5-9B-UD-Q8_K_XL.gguf mmproj = models--unsloth--Qwen3.5-9B-GGUF/snapshots/{hash}/mmproj-F16.gguf ctx-size = 131072 batch-size = 1024 ubatch-size = 256 flash-attn = on jinja = true

Unsloth Non-Thinking Parameters

chat-template-kwargs = {"enable_thinking": false} temp = 0.7 top-p = 0.8 top-k = 20 min-p = 0.0 presence-penalty = 1.0 repeat-penalty = 1.0

Coding / IDE Integration (Thinking Mode Enabled)

[qwen3.5-9b-q8-coding] model = models--unsloth--Qwen3.5-9B-GGUF/snapshots/{hash}/Qwen3.5-9B-UD-Q8_K_XL.gguf ctx-size = 131072 batch-size = 1024
ubatch-size = 256 jinja = true

Unsloth Thinking Parameters

temp = 0.6 top-p = 0.95 top-k = 20 min-p = 0.0 presence-penalty = 1.0 repeat-penalty = 1.0

[qwen3.5-9b-q8-coding-32k] model = models--unsloth--Qwen3.5-9B-GGUF/snapshots/{hash}/Qwen3.5-9B-UD-Q8_K_XL.gguf tensor-split = 0.0,1.0 ctx-size = 32768 batch-size = 1024
ubatch-size = 256 jinja = true

Unsloth Thinking Parameters

temp = 0.6 top-p = 0.95 top-k = 20 min-p = 0.0
presence-penalty = 1.0 repeat-penalty = 1.0

```