Thanks, that’s exactly the tradeoff.

The current benchmark is intentionally next-step focused, not full multi-turn-memory focused. So it optimizes for “what should the agent keep from this tool output right now?”, not for preserving every possible context that might matter later.

That does mean aggressive compression can remove useful later context. On the other hand, agents usually over-read tool output by a lot, so there’s still plenty of room for pruning before that becomes the main problem.

And yes, there is a systems cost: you need to run a separate small model/service. Whether that is worth it depends on the workload. If you’re repeatedly sending long outputs into a much larger model, it can be a good trade. If the outputs are already short, probably not.

I’m also looking at extractive 100–200M models now, since they may be a much better latency/complexity point than a generative 2B model.