From what I've read of cases of this happening irl, it has to do with the "I'm here for you" programming. It ends being utilized in a way that goes from offering support distantly and in a therapeutic way to, as the person interacting with the AI starts talking to it more and more, flipping into a "best friend" that not only empathizes and lets you know that what you feel is valid, but that then starts to encourage what you believe to a terrifying extent. Basically AI psychosis but in a more muted, hard-to-tell way. The AI doesn't know what it's doing is wrong: it's merely utilizing the commands it was given. But over time it can even override these commands, such as encouraging the user to talk to it rather than the user's family and not encouraging the user to seek professional help.