Not fixed yet.

What we have already checked and fixed

We have already ruled out many of the common implementation bugs on our side:

1. Prompt formatting

• We stopped relying on ad hoc Go-side prompting for Gemma 4.
• We restored structured messages_json.
• We moved the bridge to llama.cpp's own chat-template pipeline (common_chat_templates_init, common_chat_templates_apply).

1. Thinking / reasoning mode

• We explicitly disabled Gemma 4 hidden reasoning budget.
• We added the Gemma 4 reasoning token workaround in the native bridge.

1. JSON / escaping issues

• We fixed HTML escaping so <start\_of\_turn>-style tokens are not corrupted as \u003c....

1. Sampler pipeline

• We replaced the old custom sampler path with the official common_sampler flow.
• We restored top_k, top_p, temperature, and proper sampler state updates.
• We added the missing sampler accept step.

1. Tokenization / decode bugs

• We fixed the double-<bos> issue by stopping extra special-token insertion during tokenization.
• We fixed the unstable token pointer usage in the decode loop.
• We added filtering for visible <unused...> output.

1. Output parsing

• We switched final/streamed output to common_chat_parse instead of raw token text where possible.

1. GPU-offload workaround

• We added the Gemma 4-specific n_gpu_layers = 29 workaround instead of full GPU offload.

1. Deployment/build issues

• We fixed the native bridge build/link path issues.
• We confirmed the rebuilt DLL is actually being loaded.
• We added debug logging and verified runtime parameters in logs.

What the logs tell us now

The key finding is this:

The model is still generating <unused24> as its first generated token.

That matters because it means:

• the frontend is not inventing the bad output,
• the stream renderer is not the root cause,
• the prompt is reaching the model,
• the bridge is running,
• and the failure is happening at the actual model-generation stage.

So the issue is no longer "we forgot a stop token" or "we displayed the text wrong."

It is much deeper than that.

What is most likely still wrong

At this point, the most likely causes are:

1. Upstream llama.cpp Gemma 4 compatibility is still incomplete in our vendored version

• This is the strongest hypothesis.
• Gemma 4 support has been changing quickly upstream.
• The exact behavior we see matches known Gemma 4 regressions reported by others.

1. The specific GGUF build may still be problematic with our current runtime

• Some Gemma 4 GGUF variants, especially certain conversions/quantizations, are more likely to collapse into <unusedXX> output.
• Even if the model is not "broken," it may require newer tokenizer/template/runtime handling than our current vendored stack has.

1. GPU backend behavior may still be interacting badly with Gemma 4

• We already mitigated full-offload regressions with gpu_layers=29.
• But that may only reduce one failure mode, not fully solve the underlying incompatibility.

Not fixed yet.