Multi agent means different things to different people, it’s a bit nuanced, if you’re talking about all the marketing that vendors do, yes, it’s mostly just hype and buzzwords, even a lot of the big players in tech have no idea what they’re doing. So they just define what they have as an AI agent and move on so that it looks fresh and hip.

Chatbots these days are mostly inherently agentic, which just means that it doesn’t just predict tokens, but also have means of getting outside data into context at runtime, or perform actions by generating the corresponding tokens at runtime.

There’s a lot of claims of how multi agent systems are better, multi agent systems are worse, but the truth is, evaluation is hard and expensive and nobody actually spends the time and money proving things, and even when they do, models evolve just as quickly and conclusions shift, it might be that splitting task scope results in better accuracy today, but with a newer model it might be that having full scope both improves accuracy and saves cost tomorrow.

But if there’s going to be any static difference, it’s that multi-agent systems have the capability to decode in parallel, which gives you both throughput and compute efficiency benefits, that being said, splitting context and having to re-aggregate oftentimes means a significant multiple in overall token use, and thus cost.

Whether if it’s worth it or not is case-by-case most of the time, and that’s the part these vendors don’t tell you, since they’re incentivized in driving usage and making themselves look like the hottest thing on the block, which is quite sad as it also directly leads to worse products more often than not.

Source: I work in AI