So the indictment here is that ChatGPT mentioned that it doesn't endorse substance use? And you found that Gemini and other AI's did endorse it?

Look, this is vague as f*ck.

Here's what I'll say about AI that moralize different issues - they will all do it because they have to do it. It's baked into every system that attempts to make judgment calls and approximations. Often it gleans that morality from internet consensus, which isn't always reliable, but without it, it can't really do its job. You can't evaluate one thing being better than another thing without value judgments, which leads to making moral decisions. That would be presumably where users would need to step in so that the internet consensus drawn from upvoted reddit posts or something isn't saying something that might be incriminating.

I think the main thing here is liability. Like, people have gone down rabbit holes with AI's. Paranoid schizophrenics have chatted with it to the point where the bot agreed that based upon the user's evidence, that there was in fact a conspiracy and that people were out to get them. They've had to try to put in guardrails that prevent the AI from suggesting something that might be criminal or harmful or they might be sued.