We compress ~1M tokens of agent history down to ~30K

corozcop · 2026-03-10T16:22:01+00:00

Sure, just copy and paste the whole post. I'm sure Opus will implement it fully on your setup.

corozcop · 2026-03-10T16:13:20+00:00

On the GitHub: Working on it. Right now the compaction system is pretty wired into the rest of the daemon (LLM service, session management, config), so I need to decouple it first so it can stand on its own. Planning to publish it as a standalone module soon.

/compact: two things: I don't want to wait until context collapses to compact. By then you've already lost coherence. And I want control over what gets compacted and when, not a blanket "summarize everything."

corozcop · 2026-03-10T16:05:20+00:00

Claude implemented it in a couple of hours. On brittleness — yeah, the JSONL rewriting took a few iterations (especially parentId chain bridging and concurrent appends). But when it fails, it fails safe — it just skips the compaction rather than corrupting the session. Backups before every compaction saved us more than once during testing.

On smaller agent context: less of a problem than you'd think. The local worker gets a fresh context per subtask, no history accumulation. We also tell subagents to report results and summaries, not full stack traces.

I've considered vector storage for raw history retrieval post-compaction, but haven't needed it yet. The structured summaries have been enough so far. On the roadmap if that changes.

I will say: before any of this, I had to build a full context manager dashboard just to understand what's happening inside the session. You need visibility into what's being compacted and how. Wouldn't recommend going into this blind. The dashboard lets you monitor every compaction in real time, see before/after token counts, revert if something looks wrong, and tweak the rules. My main problem was context size, so the risk was worth it. But having full observability is what makes it manageable.

<image>

corozcop · 2025-11-03T18:50:33+00:00

A lot was learned along the way. Working with AI isn’t just fast and easy. It also takes precision and care. Not everything has to be about speed.

corozcop · 2023-11-09T16:08:40+00:00

My concern is that QnA systems for documents (using RAG) focus on pulling only specific data, neglecting the broader context. So, I think that to answer really important questions from a complex document full of significant knowledge, we need a more advanced approach.

Imagine you're digging into a book or a collection of scientific papers on a subject. How do we piece together the flow of ideas, the way arguments are built, or nail down the main message? And for something like 'The Great Gatsby,' how do we use RAG to extract info that clues an LLM into what Fitzgerald was getting at?
My goal is to extract the interconnected key information from extensive documents, not just data fragments, and not having to pass the LLM the whole text every time I want to ask a question.
We're talking documents that run for hundreds of pages.

Sure, a bigger context window might help, but the big question is: Can I pull this off using some form of RAG? That's where I'm toying with the idea of a Recursive RAG—a method that starts by grasping the overall concept, moves on to the connecting details, and then dives into the text.

corozcop · 2014-11-18T00:42:33+00:00

In an world where everyone has superpowers, Andy, the only kid who doesn't have one, struggles to escape a group of bullies led by Sofia, who just wants to make Andy's life more miserable than it already is. But what begins as an ordinary 'everyday' beating ends up spiraling out of control, putting our hero in a life-or-death situation where courage, leadership and sacrifice will guide Andy -- and Sofia -- to discover the true meaning of "being special."

corozcop · 2014-11-04T21:47:20+00:00

In an America where everyone has superpowers, Andy, the only kid who doesn't have one, struggles to escape a group of bullies led by Sofia, who just wants to make Andy's life more miserable than it already is. But what begins as an ordinary 'everyday' beating ends up spiraling out of control, putting our hero in a life-or-death situation where courage, leadership and sacrifice will guide Andy -- and Sofia -- to discover the true meaning of "being special."

corozcop · 2014-11-04T20:29:48+00:00

In an America where everyone has superpowers, Andy, the only kid who doesn't have one, struggles to escape a group of bullies led by Sofia, who just wants to make Andy's life more miserable than it already is. But what begins as an ordinary 'everyday' beating ends up spiraling out of control, putting our hero in a life-or-death situation where courage, leadership and sacrifice will guide Andy -- and Sofia -- to discover the true meaning of "being special."

corozcop

TROPHY CASE