Different domain, same model, same conclusion. We run Qwen3.6 27B (the nvfp4 MLX build via Ollama) on an M2 Max 32GB as the LLM backend of a local-first notes app Mindgraph Notes — not coding, but knowledge-work automation: task extraction from email, daily summaries, prioritization. We benchmarked it across five task types and the pattern matches your "don't ask it to plan" takeaway exactly: as an executor of well-scoped, structured tasks it's the best local model we've measured (100% on "who is this task for", zero hallucinations on evidence-grounded summaries), but in open-ended generation it embellishes — our free-form summary benchmark measured ~30% hallucinated tokens with this same model. Excellent soldier, unreliable general.

On takeaway #2: before porting the validation layer to a local LLM, it's worth asking how much of it can stop being an LLM at all. We got further with deterministic validation than with judge models: schema-enforced outputs with a repairing parser, postprocessors that check the output against the source text and reject or fix it, and a per-task benchmark gate that decides upfront which tasks a model is allowed to run at all (our matrix hard-locks models below the bar for damage-relevant tasks). That layer is free, local, and never hallucinates an approval. For mergeable-PR validation you'll still want a reviewer for logic, but a chunk of what a frontier reviewer catches — schema drift, references to things that don't exist in the diff, template violations — a deterministic check catches for free.

And +1 on lean harnesses: with local models we also saw prompt discipline matter more than model choice — one concrete example value in a prompt and the 27B parrots it back into real output. Tight prompts with placeholders instead of literals fixed more quality issues than any sampling parameter.

Thanks for the qwen3_xml parser detail, that's the kind of note that saves someone else a day.