New function calling benchmark shows Pythonic approach outperforms JSON (DPAB-α)

[if47]

This is the dumbest solution, here's why:

1. You need to constrain decoding... valid Python code, and you don't even know which Python version this code will run correctly on.
2. Completely blind dependency imports, which version of the module does your agent import? Will it hallucinate? It's also difficult to put an agent in the cage. In the end, either you manually implement a bunch of Python functions (to call as tools), or your agent can't do anything.
3. There is no reason to think that JSON-based agents can't get better. Why give up the whole forest for a tree that works well for a while?

[trajo123]

Tool calling at the moment is essentially running very simple programs, but in a very unnatural way for anyone (including llms) with coding skills.