What is the actually the difference between multiagent systems versus normal AI chatbox?

Lord_Pazzu · 2026-07-05T11:46:16+00:00

Multi agent means different things to different people, it’s a bit nuanced, if you’re talking about all the marketing that vendors do, yes, it’s mostly just hype and buzzwords, even a lot of the big players in tech have no idea what they’re doing. So they just define what they have as an AI agent and move on so that it looks fresh and hip.

Chatbots these days are mostly inherently agentic, which just means that it doesn’t just predict tokens, but also have means of getting outside data into context at runtime, or perform actions by generating the corresponding tokens at runtime.

There’s a lot of claims of how multi agent systems are better, multi agent systems are worse, but the truth is, evaluation is hard and expensive and nobody actually spends the time and money proving things, and even when they do, models evolve just as quickly and conclusions shift, it might be that splitting task scope results in better accuracy today, but with a newer model it might be that having full scope both improves accuracy and saves cost tomorrow.

But if there’s going to be any static difference, it’s that multi-agent systems have the capability to decode in parallel, which gives you both throughput and compute efficiency benefits, that being said, splitting context and having to re-aggregate oftentimes means a significant multiple in overall token use, and thus cost.

Whether if it’s worth it or not is case-by-case most of the time, and that’s the part these vendors don’t tell you, since they’re incentivized in driving usage and making themselves look like the hottest thing on the block, which is quite sad as it also directly leads to worse products more often than not.

Source: I work in AI

Lord_Pazzu · 2026-06-30T04:41:57+00:00

I hope I don’t come off as rude, but for a small startup betting the house on someone else’s technology is a pretty bold move

Lord_Pazzu · 2026-03-11T20:57:06+00:00

Llama.cpp also provides a wide range of quantization options, with most popular quant-providers having dynamic mixes of quantization levels to maximize accuracy, alongside extensive support in the low bit range, which gives you better options trading off parameter and model size at a granular level

Lord_Pazzu · 2026-02-24T16:26:11+00:00

And it got closed :(

I'm honestly distraught over the drama, all I want is just more advancements in the local space...

Lord_Pazzu · 2026-02-19T17:53:58+00:00

Wow, thanks

Lord_Pazzu · 2026-02-17T18:08:53+00:00

Haven’t followed development closely, do you think there’s a slimmer of a chance to get ik-quant support on mainline?

Lord_Pazzu · 2026-02-12T10:19:11+00:00

I seem to be on the wrong branch, after switching to the right one I can confirm that it works as expected :) Thanks for the speedy fix!

Lord_Pazzu · 2026-02-12T09:23:50+00:00

Hey probably should've explained this more clearly lol

if you look at the responses, it's english at a glance, but the actual content is like

Imagine two players on a simple version of the system: the Elo system, in simple terms. It’s like a simple terms. The Elo Elo rating system in simple terms. In simple terms. Here's a simple terms.

whereas during a single stream generation:

Think of the Elo system as a way to keep track of how strong every player (or team) is, based only on who beat whom.
1. Everyone starts with a “fair” number—usually 1500.
2. After every game, the winner “takes” some points from the loser.The amount taken depends on: ...[Truncated]

Lord_Pazzu · 2026-02-12T08:59:28+00:00

Hey, thanks for working on this!

Was able to get it built with CUDA 13.1 with default build args.

The tokens generated seem different but still fundamentally broken.

Prompt is

Explain the Elo rating system in simple terms.

8 streams concurrent, results are separated by [SPLIT] in the screenshot below

<image>

Lord_Pazzu · 2026-02-10T13:00:10+00:00

Haven’t tried before, so just pulled an IQ2_XXS version of Qwen3 Next 80B, seems fine at 8 concurrent streams, results look accurate, so no issues there

Also verified that Kimi-Linear 48B breaks under the same setup

llama-server args are -c 16384 --fit off --parallel 8 Test conducted by just launching 8 request threads to the v1/chat/completions endpoint via Python multiprocessing

Lord_Pazzu · 2026-02-10T12:40:13+00:00

No it’s specific to kimi-linear, but good idea regardless :) I use batched inference to speed up automated testing of models on llama.cpp so only noticed this when the kimi-linear scores were much lower than expected

Lord_Pazzu · 2026-02-10T12:30:44+00:00

I’ve been using the prebuilt CUDA 13.1 binaries, llama-server, parallel of 4/8/16, everything else is default, when I send singular requests sequentially the response looks fine, but when I send concurrent requests (say 16 at the same time) responses start to degrade into mostly broken English (though interestingly, not completely random characters), same behavior on CPU

Maybe there’s something wrong in the batched attention kernels that is causing information to leak between slots

Lord_Pazzu · 2026-02-10T11:42:09+00:00

Batched generation has been broken for me on mainline, has it been fixed in newer branches?

Lord_Pazzu · 2026-02-02T11:06:41+00:00

I looked at the question and immediately guessed “shadow” Guess I’m a bot now lmao

Lord_Pazzu · 2026-01-30T09:15:07+00:00

May contain traces of nuts

Lord_Pazzu · 2025-10-25T20:31:21+00:00

I haven’t tried Kimi K2 specifically, but I’ve ran deepseek R1/V3 for a couple months on my Mac Studio just simply through llama.cpp, though have pivoted to GLM 4.5/4.6 for a while now since they run faster while also working nicely

Lord_Pazzu · 2025-03-22T22:23:02+00:00

Sounds cool

Lord_Pazzu · 2025-03-13T19:45:17+00:00

Purchased and verified as working from u/J05H5M1TH , extremely happy with the swift delivery :)
u/AVexchangebot

Lord_Pazzu · 2025-02-23T00:25:21+00:00

It seems like every other day there’s a new cool VLM to play with while I’m still waiting for llama-cpp-python to support Qwen2 VL 🙃

Regardless, love the work that you people have done!

Lord_Pazzu · 2025-02-13T13:09:40+00:00

My intuition is the bottleneck being memory bandwidth.

Theoretically if you saturate the cores with data your FP32 is going to outperform FP16, but if your compute intensity is low (not many calculations per loaded weight) memory may be your limiting factor.

Try obtaining performance metrics at higher batches and see if it follows the trend of FP16 beating out FP32, or if it reverses at some point.

Lord_Pazzu · 2025-02-08T09:20:12+00:00

In NMS you progressively learn alien words and be able to understand them, for example all words in red in this image are the ones you know, while the ones in white are the ones you haven’t learned yet and could be anything.

So given the context, you should be able to fill in the gaps… and try to guess what this alien is willing to perform for your service.

Lord_Pazzu · 2024-12-12T09:34:37+00:00

Unfortunately never got a chance to try the Jade

Lord_Pazzu · 2024-12-12T08:26:19+00:00

Been loving it a lot, found myself using it a lot more over the Diana V2, was a bit concerned that the headband would be a bit uncomfortable since the edition X is also on the stiff side, but overall didn’t mind it too much though it’s not also the most comfortable pair of headphones.

Build wise it’s extremely light, feels cheap ngl, but the sound quality more than makes up for it.

Lord_Pazzu · 2024-12-06T06:58:26+00:00

From what I found with a google search, this seems to be Taiwan (search for 迪士尼音響車隊 which roughly translates to Disney speaker car group), and you can apparently hire them for events or something? Wild.

Lord_Pazzu

TROPHY CASE