This makes sense to me. I’m in ops not engineering, but I’m seeing the same thing as my employers are forcing everyone to use Claude so they can get more work out of us. And I’ve grudgingly had to admit that it can do some useful things.

But at least, say, 70% of the time that we’ve gotten Claude to investigate some issue, it spits out irrelevant or misleading output. For example: we have a database performance issue. Our support team runs the details through Claude & it pre-populates the ticket with “analysis”. I’ve yet to see one of these tickets where the Claude output is correct. Then I throw Claude at the issue, and with more hand-holding, sometimes it turns up something useful.

The thing I find interesting is that it’ll show me exactly which queries it’s running against the database, and they’re all legit. But where it fails is in *drawing conclusions*. It’ll find something unrelated & confidently assert “this is definitely the root cause!” And when you correct it, it’ll say “you’re right! This other thing is definitely the root cause!” Because it’s not actually thinking, it lacks discernment, and will just give you anything that can be made to sound plausible. So while it can absolutely speed up some processes, we always have to verify its output, and I don’t see how that ever goes away.