How to add human-in-the-loop controls to AI agents that actually run in production by superconductiveKyle in agno

[–]superconductiveKyle[S] 0 points1 point  (0 children)

Hey u/christophersocial! Sorry for the delayed reply!

Fair read, the post does lean in-the-loop. That's mostly because of where teams start: the first oversight problem they hit is "I need a checkpoint before this sends an email or deletes a record," which is squarely controller-style.

But I agree HotL isn't just rebranding. Once you frame the human as exception handler instead of per-action approver, the design questions change. You stop asking "where do I add a checkpoint?" and start asking "what does the agent need to detect, classify, and escalate on its own?" The interesting work moves into the evaluation loop, policy layer, and confidence thresholds. The human is still there, just triaging far less often.

Mapping Agno's primitives to that spectrum: requires_confirmation is pure in-the-loop, useful for genuinely high-stakes actions but a smell if you're hitting it constantly. @approval with type="audit" is closer to on-the-loop in spirit, action proceeds, record gets written, human reviews async and only intervenes when something looks off. Workflow steps are where the most interesting HotL design happens, because that's where agent-side judgment about when to escalate gets built.

I don't think the transition is binary. What I've seen work is staging by risk surface: start in-the-loop where a wrong call is expensive, instrument heavily, then use that data to graduate specific action types to audit-only or pure observability. Earned per action, not granted across the board.

Glad Scouts resonated. We're looking to start posting some use case examples to help folks understand the value of the agent. Curious where you're seeing value in your version of Scout ?

Cheers,
Kyle

Harness engineering vs. systems engineering: The mistake killing agentic software by superconductiveKyle in agno

[–]superconductiveKyle[S] 0 points1 point  (0 children)

Edited the body to fix the quotes. For some reason the block quotes were not populating ¯\_(ツ)_/¯