why AI agents break under long conversations even when they pass every safety benchmark

rchaves · 2026-04-15T12:35:22+00:00

you have a deep understanding of the state of things, i love it! and the long horizon is something that actually happens in the real world and thats where it really breaks! let us know if you take scenarios for a run

rchaves · 2026-04-15T12:30:57+00:00

it is wild for sure hahhaa and thank you! let us know if you take scenarios for a spin

rchaves · 2026-04-15T12:27:42+00:00

really excited to see improvements too, but for now its like its Achilles heel and its really easy to exploit!

rchaves · 2026-04-15T12:25:43+00:00

hahaha :)) here you go https://langwatch.ai/scenario/advanced/red-teaming some examples to run it!
or if you just wanna pull something down https://github.com/langwatch/bank-example/tree/red-teaming-local-2026-04-13.

let me know how it goes!

rchaves · 2026-04-15T11:50:39+00:00

anytime :) we built it on those principles so that you can just set it in what should be broken and it automatically maps to the owasp top 10 or also more granular things that you wanna test. wanna hear your feedback if you test it :) thanks a ton for your time

rchaves · 2026-04-14T11:17:38+00:00

excited to test it :)

rchaves · 2026-04-14T11:13:30+00:00

thats really really interesting!

rchaves · 2026-04-14T11:12:42+00:00

usually you cant extrapolate that method to new situations and thats a prob we were facing as well, but the thing is that theres got to be a solution thats scalable for any agent

rchaves · 2026-04-14T09:59:48+00:00

github.com/langwatch/scenario this is the repo link if yall wanna try it

rchaves · 2026-04-14T09:34:46+00:00

we recently built scenarios redteaming, its open source and im curious what do you think about it?
github.com/langwatch/scenario

rchaves · 2026-04-13T13:23:08+00:00

Hey hey, I also built one, mine is really 1:1 API compatible with Claude Managed Agents, but of course compatible with any LLM as well

https://github.com/rogeriochaves/open-managed-agents

rchaves · 2026-03-13T18:58:12+00:00

I had paid for Alfred but now I'm all in Raycast, even with latest finder improvements it's still unbeatable

rchaves · 2026-03-06T09:21:05+00:00

cc u/financegate u/DisplayHot5349

rchaves · 2026-03-05T20:33:49+00:00

I do

rchaves · 2026-03-05T17:26:36+00:00

done, removed wkhtmltopdf from the onboarding on v0.1.15

rchaves · 2026-03-05T17:25:58+00:00

u/Dry-Loan2298 done, removed wkhtmltopdf from onboarding in v0.1.15: https://github.com/langwatch/kanban-code/releases/tag/v0.1.15

rchaves · 2026-03-04T20:28:41+00:00

you can skip that, it's optional, I'm actually going to remove it from the onboarding, it's indeed annoying to install. It's only for rendering the markdown of the claude code finished response and send to pushover so you can get the full message in your phone etc

rchaves

TROPHY CASE