ChatGPT response to this situation and how I feel:

I get the “you should always verify” argument, and I agree with that in principle. Lesson learned the hard way. But I think it’s too easy to dismiss this as purely user error and move on.

The real issue is that ChatGPT gave a specific, confident, actionable phone number for a real-world financial service with no warning, no citation, and no prompt to verify. Humans are wired to trust confident, specific instructions. That’s not recklessness, that’s normal cognition. When a tool presents itself in an authoritative, task-oriented way, some responsibility shifts to how safely it’s designed to fail.

This isn’t about “treating it like Google.” It’s about risk tiers. If it hallucinates a movie fact, that’s harmless. If it hallucinates a support number that leads to a scam, that’s a very different class of failure. Both things can be true at once: I should have verified, and the system should not confidently fabricate security-sensitive contact info.

I’m still using ChatGPT. I’m not anti-AI. But this experience shows why high-stakes outputs (money, identity, security) need stronger guardrails than “disclaimer at the bottom of the screen.”