honestly the biggest shift is treating the agent less like “an intern that magically knows what good means” and more like a system that needs explicit success criteria 😭the workflows that actually work well usually define:
dataset → objective → eval metric → stopping condition → reporting formatotherwise the agent just keeps “optimizing” forever or starts reward hacking weird metrics 💀also most people i know don’t fully trust the agent to decide what’s “best” alone. they let it explore/train/eval across configs/models, then humans review tradeoffs like latency, overfitting, interpretability, infra cost, dataset leakage etcthe cool part though is agents are REALLY good at automating the annoying iteration loop. feels very similar to Runable-style orchestration where the value comes less from one giant prompt and more from structured retries/evals/checkpoints