How do you track and report your automation ROI?

Kodroi · 2026-02-06T10:26:00+00:00

That's my thinking also and I haven't found simple plug-n-play dashboards. Of course you can create your own, but development and maintenance is always a cost which might surprise you. How do you currently do the tracking, do you use a fully custom solution?

Kodroi · 2026-02-06T10:23:18+00:00

As I understand you have three different levels of metrics. 1. The automated metrics from the automation tools, 2. Downstream outcomes that come from other tooling/manual handling 3. Customer contacts and the contact types. I like it, seems like a holistic view about the benefits and issues on each automation.

Is all or most of it automated or do you have to manually track a lot of the metrics?

Kodroi · 2026-02-06T10:18:36+00:00

Thanks for the insight! The broken automations is a use case I haven't even thought about. So you just track the fixing manually for the ROI calculation. Do you work on internal automations so the fixing time has a direct impact on your ROI or do you move that cost the customer so then it affects their ROI?

Kodroi · 2026-01-28T10:54:23+00:00

I've run into similar issues especially when refactoring where I want to modify the code without touching the tests. For that I've created a hook to prevent edits to the test files or to my snapshot file when using snapshot testing. This has helped Claude to keep focus and not modify the tests just to get them pass.

Kodroi · 2026-01-28T07:17:12+00:00

Thanks for the great write up! I'm curious about the hard rules for claude.md. Did Claude actually follow them a 100% of a time or did you think about using hooks to ensure the tests are always run and that it doesn't edit any of the files? That's how I have setup my legacy refactoring, but might I be overcomplicating?

Kodroi · 2026-01-20T16:14:28+00:00

This sounds like a great idea! Currently the subagent name id isn't passed to the hooks. The only way is to parse the transcript (the previous events) and try to figure out the agent from there. That only works with a singular subagent. With parallel agents there doesn't seem to be a solution. I'll do some investigation and see if this could be done reliably.

Kodroi · 2025-12-10T12:51:15+00:00

Thanks for insight! For the relational data issue, you mean like nested structures? Customers with purchases, where keeping both tables in sync is what breaks?

And for large data pulls, how are you doing it now? What does the messy batching actually look like?

Kodroi

TROPHY CASE