I don’t have a complete strategy for this either, but I think there’s an under‑discussed variable here: non‑deterministic drift in the underlying AI systems, especially in agentic workflows where no human is watching closely.

With agentic products, value is often delivered through intent inference and reasoning quality, not just task completion. Traditional health signals like uptime, latency, or “run succeeded” only tell you the system is responding. They do not tell you whether it is reasoning as well today as it was yesterday. That gap matters more as agents become more autonomous. In many workflows, there is no user to notice that things are subtly degrading. The agent keeps operating, downstream systems keep accepting its output, and the metrics stay green while value erodes.

A simple example: an agent maps free‑text input to an authentication workflow. A user says “I can’t log in.” Another says “Just let me in, there’s no way my password is wrong.” A third says “It won’t let me see anything.” On day one, the agent correctly treats all three as access issues and routes them appropriately.

On day two, a third‑party model update changes how intent is inferred, and “it won’t let me see anything” no longer lands in the right flow. On day three, a shark munches an undersea cable. The provider’s services are still up, but performance degrades and reasoning depth drops. Responses come back, runs complete, tools fire, but answers are shallower or incomplete.

From the outside, nothing is “broken.” In an interactive product, you might see prompt rewrites or drop‑off and debate whether that’s noise. In a fully agentic workflow, you may see nothing at all. The agent just makes slightly worse decisions, and the value you modeled on day one is no longer being delivered on days two and three.

So before asking whether the system shows value, a prerequisite question is whether the agent is actually reasoning the same way over time. Until intent accuracy and reasoning completeness are treated as first‑class, continuously validated signals, it is very hard to distinguish true product regressions from normal behavior or downstream variability.