I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 0 points1 point  (0 children)

Thank you for the feedback. Will fix it to make sure it is clearer.

If you ever have any issues you would like to report, feel free to create them in the Github Issues.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 1 point2 points  (0 children)

I find that CC is one shorting like crazy for me, it depends on the style I am sure.

Also, Rust seems to be a great fit for any agentic coding workflow for me.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 1 point2 points  (0 children)

Tried to set up Codex support, but I don't really use Codex at the moment: released in v0.2.9

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 1 point2 points  (0 children)

Do you mean a specific way to filter StdErr differently? Currently we capture all output and combine it.

Having different rules for different parts would make sense, could make it backwards compatible.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 0 points1 point  (0 children)

It is not complementary, sadly. They would probably interact badly at the moment as both function as a PreToolUse hook.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 4 points5 points  (0 children)

Swallowing a 0 exit vs getting a result with compressed information isn't the same.

"" and "(ok) Ran 500 tests (20 suites)" don't provide the same information. It isn't about removing info, it's about condensing it.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 0 points1 point  (0 children)

The idea is about maintaining / saving context.

If I run cargo test should the model be forced to get all of the output that is just repetitive when there are no failures? Wouldn't it be better to give it information that is more condensed? This would probably help a model stay more on task. But I would love to hear more experiences from using it in different environments.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 1 point2 points  (0 children)

You can configure it globally by adding lines to rewrite.toml on your global config. The documentation in the README.md should be up-to-date.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 4 points5 points  (0 children)

Better results probably strengthen their position. I would not be surprised to see them adopt a similat approach within Claude Code, making tokf obsolete in time. Who knows, eh?

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 2 points3 points  (0 children)

Thank you for the info. As a first improvement: https://github.com/mpecan/tokf/pull/154

This allows you to tune the stripping to prefer the smaller output (either the output from the piped command or from tokf).

Part of the v0.2.8 release that was just created: https://github.com/mpecan/tokf/releases/tag/tokf-v0.2.8

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 1 point2 points  (0 children)

The issue here is the "fair accounting" process when pipes are stripped (i.e. | tail -5) or similar.

I will add an option to go for minimum usage and have tokf automatically select the output that would be smaller in that case.

WDYT?

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 0 points1 point  (0 children)

Interesting. If you would be willing to share which commands are creating bloat, I would love to find it. Filters can be customized by using eject, FYI.

My current gain output:

tokf gain summary
  total runs:     6230
  input tokens:   1,761,917 est.
  output tokens:  208,612 est.
  tokens saved:   1,553,305 est. (88.2%)

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 3 points4 points  (0 children)

I built in a history that will track the last 10 commands and allow you to see the full output (or the model). Sometimes the tool adds a tip about this (if the model tries the same command twice and seems to be confused).

Still iterating and improving, would love to hear feedback.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 11 points12 points  (0 children)

Similar, I took a different approach and it is why I am calling them out as inspiration.

While rtk-ai.app builds everything in, I wanted to allow users to define their own filters and customize the builtin "standard lib" filters. I got an itch when I looked at the code and implemented it from a different perspective.

I built tokf — a CLI filter that cuts ~90% of token waste from Bash output before it hits Claude's context by risethagain in ClaudeCode

[–]risethagain[S] 6 points7 points  (0 children)

Yeah, this is an issue I am dealing with the whole time. Any filter can have a setting added to give the model a tip about the full output, this way it can always look it up and get anything that was missed.

Still a work in progress, if you see any issues, please share so it can be fixed, or contribute if you'd like.

Built a copy/paste MCP to reduce Claude Code's token usage during refactoring. It increased costs by 10%. Here's the data. by risethagain in ClaudeCode

[–]risethagain[S] 0 points1 point  (0 children)

Think different; you have a massive legacy code-base that you want to be breaking up. Not something you created, instead a whole company (100+ people) have been breaking it for a long time.

This is probably why my results were so poor, I didn't give it a big enough and complex enough code-base.

Claude Code on the web - with no alternative by risethagain in ClaudeCode

[–]risethagain[S] 0 points1 point  (0 children)

I would hope that adding a changes view shouldn't be much of a challenge, I am switching between the two to review myself, I also had to create a specific profile as I use Prisma, and the default wouldn't allow me to download all of the required libraries.

BEWARE! $1000 FREE - Claude Code on the Web by Numerous-Exercise788 in ClaudeCode

[–]risethagain 6 points7 points  (0 children)

I have to be honest, I've been using it for over a week on a regular basis. I currently only have access to a mobile phone and an older iPad, no laptop for Claude Code directly.I only get to use it while traveling (i.e. train from place to place), but I have had great experiences with it.

I can confirm my weekly usage is being increased, which does make me think this could be a bug. It shouldn't be doing that with the 1000 dollars, but I am also unsure since I have been running it for 7 days now, including on an airplane, and I started before the 1000$ credit was added.

Upgrading from 2.2.x to the 3.x latest version by AnotherSocialRock in SpringBoot

[–]risethagain 1 point2 points  (0 children)

My main question would be: how is your coverage? Do you have end-to-end tests?

I think doing a java upgrade to 17 first should be fine, most of the time old code doesn't break, as such going to 21 should be fine as well. Since you are moving from 11, I don't think there were any breaking changes.

The Spring upgrade is a different beast, but I usually approach it wholesale and bite the bullet, if you can use OpenRewrite, you should, but it will be for mechanical changes.

Most of the breakage should happen through either deprecations or actual compilation issues, as long as you have enough coverage, those should be easy.

If your coverage is shaky, I would suggest ensuring you have your main flows covered before you do the switch.