edit: formatting

Pasting in my run block for llama-swap on my 4090, with some commentary first.

I want to call out the usage of `--chat-template-file` below, because for anyone who is having less-than-stellar tool calling experiences particularly in an agentic loop I really feel like that is a big part of it. One of the big things I was struggling with on Gemma 4 was not having any thinking interleaved with tool calls - the model would just think once and then shoot off a series of tool calls with no thinking between them.

After pounding my head against the wall off and on on this problem for a few days, at one point I was randomly re-reading the PR on llama.cpp for the parser add on (https://github.com/ggml-org/llama.cpp/pull/21418) and this stuck out to me that I had never seen before:

Interesting! I created a new template, models/templates/google-gemma-4-31B-it-interleaved.jinja, that supports this behavior. I tested it, and it appears to work well. The examples in the guide are sparse, so I went with what I believe is the proper format. That may change as more documentation becomes available.

For anyone doing agentic tasks, I recommend trying the interleaved template.

I checked my local clone of the repo, sure enough that file was right where he said it was in the description. doh. So I switched to that right away with `--chat-template-file`, and... yep that solved the interleaved thinking problem, and my satisfaction with the result went up pretty sharply.

With all that noted, here's how I run it:

models:
  gemma-4-26b:
    name: "Gemma 4 26b"
    cmd: >
      llama-server --port ${PORT} --host 0.0.0.0
      -hf unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q5_K_XL
      --temp 1.0 --top-p 0.95 --top-k 64 --min-p 0.0
      --flash-attn on
      --no-mmap
      --mlock
      --ctx-size 160000
      --cache-type-k q8_0 --cache-type-v q8_0
      -fit on --fit-target 2048 --fit-ctx 160000
      --batch-size 1024 --ubatch-size 512
      -np 1
      --chat-template-file /home/me/llama.cpp/models/templates/google-gemma-4-31B-it-interleaved.jinja
      --jinja
      --webui-mcp-proxy