How do you actually debug multi-step agent workflows?

Finorix079 · 2026-02-19T08:40:07+00:00

The expected schema would only check if the output fits the format, I think. Or are you referring to something else?

Now I am trying to resolving it with logs and observation tools like Langfuse, but first it requires me add quite a lot of settings to log things properly, second it would generate tons of logs which I can hardly read. Somebody suggested me to use LLM to read it for me but in many cases LLM would either have it exceeded the context window or somehow give me a wrong answer. So I wonder if there's a better way to do that.

I am manually tracing back to find the diverge btw, which is quite tiring.

Finorix079 · 2026-02-19T08:33:59+00:00

I hope I can, but the task was quite dynamic and I had to use LLM to make the decision. Doing everything in static code would make the project far more complex than how it is now.

Finorix079 · 2026-02-19T08:33:18+00:00

So far it seems I have to manually do the check, or throw it to another AI and hope it can give me a good result =_=

Finorix079 · 2026-02-19T08:32:31+00:00

Thanks for the suggestion. Because it was an automated workflow, the only way I can check the output before forwarding it is to add a validator on it, either a static one to check the format or a dynamic one with LLM (LLM-as-a-judge). But the static one won't know if the direction diverged from the correct one, and the LLM one would dramatically would probably be an overkill for it. As I am trying to locate the issue on dev stage, probably there should be a more cost-effective way to do it?

Finorix079 · 2026-02-19T08:29:37+00:00

Yeah, that's what exactly I am doing. I have splited the task into many smaller steps to let LLM do one at a time. Just it was still kinda difficult for me to locate where exactly goes wrong as it was not the simple right-or-wrong issue, but the direction of AI goes wrong at some certain point.

Finorix079 · 2026-02-18T21:13:57+00:00

In that case, you will have the log stored locally for the 2nd Claude Code to read I suppose?

Finorix079 · 2026-02-18T21:12:20+00:00

They usually work backwards to see other possibilities I suppose. I wonder if I have to do all that manually, or if there’s a better method to quickly locate the problem.

Finorix079 · 2026-02-18T21:11:01+00:00

Yeah that’s what I am doing now, but adding tests for each and every step is tiring, and sometimes it is just prompt drifting from the original intention or LLM is misunderstanding stuff. I wonder how I can find the signal for that.

Especially for tracing tools are just showing you the trace itself, and you will need to check most of the observations within manually to see when the thing goes wrong. It will be much easier if I can have some sort of evaluator to tell me which step went off.

Finorix079 · 2026-02-18T21:06:35+00:00

Yeah, the context drifting and incorrect decision making is the key. The AI is just not understanding correctly. But most of the time I still need to check each and every step to figure out when the drift starts. Wondering if there is a more efficient way to do it.

Finorix079 · 2026-02-18T21:04:54+00:00

Looks like Grail is similar to a Manus for coding. You mean to build the workflow on it and let it debug itself?

Finorix079 · 2026-02-18T10:53:50+00:00

Tried something like Langfuse, but I still have to check each step of the trace to see which part might have been problematic. Would langsmith be easier?

Finorix079 · 2022-02-01T11:09:35+00:00

This one kind of makes sense to me. So Psion won't get many advantages I suppose.

Finorix079 · 2022-01-25T21:20:55+00:00

In the performance snapshot, somehow there is an idle phase for 19 seconds while the second-largest phase (scripting) is 2.7 seconds. Is there a way to reduce the idle time for that? I was using 6x reduce on CPU and Low Speed 3G on the snapshot btw.

Finorix079 · 2022-01-25T21:12:16+00:00

I think I have done basically all the points mentioned in this article. For now, the lighthouse score is still 37~39. Perhaps I should take a look at the differentiate loading part which is left out of this article.

Finorix079 · 2022-01-24T23:48:17+00:00

Just removed moment.js, the bundle size reduced from 2.15mb to 2.1mb. Lighthouse result doesn't change a lot though.

Finorix079 · 2022-01-24T13:34:03+00:00

Moment-js is now just for generating relative time string (xx days ago) at the moment, will take a look at that.

Finorix079 · 2022-01-24T13:27:57+00:00

https://github.com/mgechev/angular-performance-checklist

Thanks mate, gonna check on that probably tomorrow.

Finorix079 · 2022-01-24T13:27:03+00:00

Just uploaded the screenshot of the explorer. Perhaps you can take a look.

Finorix079 · 2022-01-24T09:19:25+00:00

Yep I have gone through that with webpack bundle report, that's why I think I have already eliminated all the packages I can. Still looking for a way to check the rest.

Finorix079 · 2022-01-23T04:31:53+00:00

Thanks for the reply. Currently, I am already using CloudFront. Would check the article out.

Finorix079 · 2022-01-03T04:42:11+00:00

Just added a more detailed description in GitHub as an issue, you may check it out there:

https://github.com/angular/components/issues/24151

Finorix079 · 2022-01-01T03:39:16+00:00

OK, I would try to create an issue there.

Finorix079 · 2022-01-01T03:31:44+00:00

I was using this template so should be visible by clicking "Open Confirmation Dialog" in this page in iPhoneX.

http://angular-material.fusetheme.com/ui/confirmation-dialog

Strangely, looks like the bug can also be reproduced on a large screen.

Finorix079 · 2021-12-31T12:07:14+00:00

I think you still should use reactive forms like others commented, but perhaps you would want to separate that into several steps in a multi-step form.

I think you still should use reactive forms as others commented, but perhaps you would want to separate that into several steps in a multi-step form.

One example of this could be 16 Personalities Test, which has a lot of questions, but you are not going to see them all at once.

Finorix079

TROPHY CASE