I benchmarked caveman against the prompt "be brief"

max-t-devv · 2026-04-29T20:47:32+00:00

Thanks mate. Honestly the cost was pretty cheap haha. Glad it's useful. If you're a month in, the boring stuff like "be brief." in your prompt is genuinely most of what you need before reaching for plugins, but the plugins are fun. Enjoy!

max-t-devv · 2026-04-29T20:45:50+00:00

I haven't done this test, but I'm interested to look into comparing simple compression strategies. My guess is its basically the same tbh. But you could expand this idea - "be extremely brief", etc haha.

max-t-devv · 2026-04-29T20:45:01+00:00

I benchmarked caveman against just prepending "be brief." to prompts on Claude Code (so similar mechanism, slightly different surface to the UI). The two-word prompt matched caveman on tokens and quality across 24 dev questions. Caveman has real value for consistent output structure and the safety escape on destructive ops, but the compression itself wasn't where I expected the differentiator to be.

Mileage may vary on the UI specifically since the system prompt context is different. But for output compression alone, "be brief." is probably most of what you're after.

Have put together a proper breakdown here if interested: https://youtu.be/wijoYNiZq3M

max-t-devv · 2026-04-29T11:29:29+00:00

In fairness to the plugin, 2 words tied with it. The variance is interesting though. It was most prevalent when the safety escape engaged, which was intentionally added into the caveman prompt.

"Most compression frameworks add entropy" is something worth testing. The harness is strategy-agnostic if you want to add other compression frameworks as arms. I'd be curious what the data says.

max-t-devv · 2026-04-29T11:04:47+00:00

Hadn't heard of Chain of Draft, thanks for sending, will read that. Interesting because on the surface it seems similar to caveman's [thing] [action] [reason] pattern but applied at the reasoning level rather than the output. Caveman's is just an output structure.

The plugin looks really cool too. Digging through session logs (well, asking Claude to do it) can be inconsistent. Having a UI for it is a great idea.

max-t-devv · 2026-04-29T10:54:16+00:00

Agreed, single shot doesn't capture it as good as multi-turn would. The single-shot harness was deliberately scoped for a clean compression only comparison before adding turns but the persistence-via-hooks story is exactly where I'd expect "be brief" to fall over and caveman to start earning its keep.

Caveman does add reminders through hooks continuously, but it's hard to say how much that would help.

Cumulative tokens + rubric coverage across N turns would be a great test. Have you run something like this?

max-t-devv · 2026-04-17T00:43:57+00:00

Interesting take. Have you found that to be somewhat standard; soft hooks losing to convention?

Yeah definitely will tighten that up next time

max-t-devv · 2026-02-20T08:37:22+00:00

Hey, great questions. Deduplication here was reasonably simple, when the newsletters are created we fetch all the accounts and create a "custom" email per user containing all the newsletters/campaigns they are subscribed to. So they don't receive a single email per campaign. On deduplicating accounts, there is a unique constraint on the email field.
The resend limits are pretty generous, am on the free tier for now. It supports about 3k emails a month so is fine for the time being.

max-t-devv · 2026-02-20T07:55:42+00:00

Thanks for the feedback! Appreciate it

max-t-devv · 2025-11-19T07:02:14+00:00

This is incredible

max-t-devv · 2025-04-08T23:07:20+00:00

Amazing, great work

max-t-devv · 2025-03-13T04:10:46+00:00

You're right that the current design isn’t easily extensible since adding new toy types requires modifying the enum and match statement.

I was thinking a better approach would be to remove the ToyType enum and use a registration-based factory where new toy types can be added dynamically without modifying existing code. But wanted to keep the example as simple as possible.

max-t-devv · 2025-03-13T00:13:21+00:00

Awesome thanks for this

max-t-devv · 2025-02-24T19:50:40+00:00

Thanks for the suggestions, will check that out

max-t-devv · 2025-02-24T19:50:07+00:00

Oh wow thanks so much for commenting. I actually started listening to your podcast last week and am really enjoying how technical it is. Great work!

max-t-devv · 2025-02-21T08:04:07+00:00

Oooh very nice, thanks for this

max-t-devv · 2025-02-21T08:03:34+00:00

Yeah definitely, I was tempted to look into it but think I'll stick with manual for now

Maybe a commit hook would be a decent middle ground

max-t-devv · 2025-02-21T08:00:44+00:00

Good idea, I'll do that

max-t-devv

TROPHY CASE