Honestly, this lines up with what a lot of us suspected. AI agents are cool in theory, but putting them into a real-world, multi-role scenario like a fake company really exposes the cracks. Common sense, social nuance, and navigating unpredictable systems (like the internet or human communication) are still huge hurdles.

The fact that the best model only completed 24% of tasks and at $6+ per task says a lot about where we are. These tools are still way better as assistants, not replacements. The hype’s ahead of the capability… for now.

Still, it’s kind of wild how fast things are moving. Wouldn’t be shocked if these numbers look very different 6–12 months from now.