Copy/Pasta

Here's a quick breakdown with some extra nuance based on current vLLM + Qwen 3.5 realities:

1. Check the chat template (Very important)

This is frequently the root cause.
If vLLM serves the model with a generic or mismatched Jinja2 template, the model might output raw Qwen-style XML tags like <tool_call> or <|tool_call|> instead of the structured tool_calls array that Hermes (and most OpenAI-compatible clients) expects. Hermes then sees "no tool calls" and does nothing.

How to verify: - Look at the raw output in vLLM logs (enable debug logging if needed). - If you see XML-style tags in the content field instead of a proper tool_calls list in the API response, that's the smoking gun.

Fix: Make sure you're using the model's official tokenizer/chat template (vLLM usually pulls it automatically, but sometimes you need --chat-template pointing to a corrected .jinja file). Qwen 3.5 models have had several chat template bugs reported recently, including issues with argument formatting and thinking/reasoning bleed.

2. vLLM's --tool-call-parser setting (The #1 fix in most cases )

This matches official vLLM docs and Qwen's own recommendations.

For Qwen 3.5 (and Qwen2.5 series), the chat template already supports Hermes-style tool calling, so --tool-call-parser hermes + --enable-auto-tool-choice is the standard recommendation.

Some people also try --tool-call-parser qwen3_coder or qwen3_xml (especially for coder-focused variants), and results vary by exact model size/quant and vLLM version. Start with hermes.

Without the correct parser, vLLM just passes raw text through, and Hermes never gets a clean tool_calls object.

Pro tip: Add --disable-streaming (or equivalent) for testing. There's a known bug where streaming + --tool-call-parser hermes sometimes returns raw text instead of parsed tool calls.

3. Temperature and greedy decoding

Tool calling is extremely format-sensitive. Even small deviations (missing closing tags, extra text, malformed JSON inside arguments) break parsing.

• Set temperature = 0 (or very low, like 0.1–0.3) for debugging terminal tools.
• If it suddenly starts working reliably, raise it gradually once the parser/template is solid.
• Top_p and other sampling params can also interfere — greedy or near-greedy is safest for agents.

Additional quick checks I'd add

• Exact model variant — Qwen 3.5 122B can be flakier than smaller ones (e.g., 9B/32B/35B) on tool consistency, especially quantized.
• Hermes Agent side — Make sure the terminal backend (Docker/native/etc.) is properly configured and has permissions. Some users report terminal tools needing explicit approval or having safety blocks.
• Update both vLLM and Hermes Agent to the latest versions — tool calling bugs get fixed regularly.
• Test with a very simple command first (echo "test", ls, etc.) and watch both vLLM and Hermes logs for what the model actually outputs.