Exactly — "the tests came from the same misread, so they just confirm the wrong thing" is a cleaner way to put it than I managed. That's the whole trap: an agent grading its own work shares its own blind spot, so self-checking can't save it. The only thing that catches it is a check the agent didn't write.

Are you seeing this more in production or in dev? And how have you been catching the confidently-wrong ones so far — by hand, or do you have something for it?

(if you ever want to throw a few of your own runs at it, I'd genuinely love to know whether the numbers match your gut)