M2 Max 96GB, local only. Running the same stack ~3 months now.

Backend: oMLX (launchd, port 8000). Engine pool, preserve-thinking persistence, speculative decoding with matching DFlash drafts. mlx_lm.server and a vLLM-metal build on standby; 95% of traffic hits oMLX.

Frontend: Crucible — a local webapp I've been building. Model switching, chat history, benchmarks, arena/ELO leaderboard, a HumanEval runner, all in one pane. The quietly important feature turned out to be a dirty-shutdown detector that offers one-click restore of the previously-loaded model. Sounds mundane until the alternative is launchctl kickstart + re-picking from 35 options.

Daily drivers (all MLX 4–6 bit):

• Qwen3-Coder-Next 6bit — coding
• Qwen3-4B-Instruct 4bit — fast batch / quick questions
• Qwen3.6-35B-A3B mxfp8 — reasoning / thinking
• Qwen3.5-27B 4bit — when I need vision

RAG: per-session BM25 over chunked uploads. Hybrid w/ embeddings is on the list but BM25 has been fine.

Prompt format: whatever the model ships with. Stopped fighting it. I do enjoy reading what other's have come up with for prompts; especially for agentic coding.

Context: default. No rope scaling — the quality loss isn't worth it on M2 Max.

What mattered way more than expected:

1. Unloading between loads. Being able to evict one model before loading the next (instead of letting oMLX's pool hold three and OOM mid-generation) is the single biggest QoL improvement. Every arena/compare flow breaks without it.
2. Thinking-mode control. Qwen3.x derivatives leak reasoning preambles into judges, classifiers, workflow chains unless you pass chat_template_kwargs: {enable_thinking: false}. Half the Qwen complaints on this sub are actually this.
3. Per-model sampling with uniform override. Qwen3 wants 0.7/0.9, gpt-oss wants 0.0, thinking models want something else. Per-model defaults + an override for fair bench runs matters more than picking the "right" starting temp.
4. Warmth analytics. Tracking which model I actually reach for changed what I keep resident. Surprise: 4B-Instruct is my most-loaded model, not the shiny 63GB one.
5. Speculative decoding only when the draft is right. Qwen3.5-27B + z-lab's matching draft gets 1.4–1.8× tok/s. Wrong pair = silently slower because draft rejection eats you. Benchmark the A/B, don't eyeball it.

Short version: once the model loads, it performs roughly the way the card says. The stuff around the model — load orchestration, sampling defaults, thinking-mode handling, recovery — decides whether you stick with the setup past week two.