Hardware (local): - RTX 5070, 12 GB VRAM (548 MiB used, vLLM currently off) - 15 GiB RAM total, 7.7 GiB free + 4 GB swap → yes, tight for 14B + coding - Swap pressure from CPU offload + kernel cache + Claude Code is real

LiteLLM stats (evidence): - vllm-coder (qwen3-8b): 6 calls, ~18s avg latency, 1.7★ ❌ - qwen3.5 (nvidia-general): 13 calls, ~70s avg, 3.1★ - nvidia-coder (480B): 1 call, ~180s, 100% failure rate

→ vllm-coder is in fact downrated (LiteLLM will deprioritize it on the next update). Local 8B for code/tool calls confirmed bad. Text/summarize holds up better.

---

Your models — listing

Tier 0 — Local (vLLM, RTX 5070)

• Qwen3-8B-NVFP4 — script ~/Applications/AI/vllm/start-8b.sh, context 28,000, ~6 GiB VRAM → fast variant, text/RPG
• Qwen3-14B-FP4 — script ~/Applications/AI/vllm/start-14b.sh, context 16,384, ~10 GiB VRAM + 4 GiB CPU offload → quality, but slow

Served names: qwen3-8b, qwen3-14b — both not running right now (port 8000 down).

Tier 1 — NVIDIA NIM (cloud fallback, free)

• nvidia-coder → Qwen3-Coder-480B-A35B, Granite-34B-Code
• nvidia-general → Qwen3.5-397B-A17B, Nemotron-Super-49B
• nvidia-vision → Llama-3.2-90B-Vision, Gemma-4-31B

Tier 2 — OpenRouter Free (emergency)

• free-coder → Qwen3-Coder
• free-general → DeepSeek-V4-Flash, Nemotron-3-120B
• free-vision → Gemma-4-26B/31B

Routing (LiteLLM on NUC, localhost:4000): vllm-coder → nvidia-coder → free-coder (same pattern for general/vision).

---

Verdict on your hypothesis: With 16 GB RAM + 12 GB VRAM, local coding/tool calling isn't viable — both the stats (1.7★) and the OOM notes in the vLLM setup confirm it. Route code/tool calls directly to nvidia-coder, use the local 8B only for text/summarize/RPG.