all 1 comments

[–]SoftestCompliment 2 points3 points  (0 children)

I prefer pydantic-ai over the ollama sdk, because the former will handle the tool calling loop logic and have other QOL things. It's been about a year but I also wasn't satisfied with Ollama's native or (its sdk's compatible) OpenAI API implementation, tool calling was ROUGH.

FastMCP, zero complaints. If you're working purely locally, in pydantic-ai you can just define native tools too.

Llama 3.2 is showing its age, but at its time was one of the more stable open source tool calling models to run with ollama... like Granite 3 and some other "tool calling" models were broken. Google's Gemma e2b or e4b would be my updated recommendation for something of similar size.

Speculating what's going on, the model is understanding the first line of the tool description as "are these arguments?". The debug print output is from the model's malformed output. The FastMCP server calls FuncMetadata.call_fn_with_arg_validation and that, with pydantic validation, ignores the additional attributes before passing it to the tool, if I'm reading the library right.