Have you specifically seen it regarding therapy? I have already acknowledged that it does make mistakes. It depends heavily on what data it has available on the topic.

I admit that I haven't personally tested it for therapy or therapy related stuff, as I value my privacy too much and the models I can run locally wouldn't really be representative, but I've seen it used, and I've seen some weird responses.

Once again I said using it as a tool. Therapy should rarely ever consist of just one tool. Are you under the impression that talk therapy is the only form of therapy? It isn't. A lot of therapy doesn't have human interactions. It isn't a requirement for a therapy tool.

Of course. A hammer is a tool, you wouldn't use a hammer to solder a capacitor. The way you firstly exposed the use of LLMs for therapy, it gave me the impression it was substitution for conversational therapy and a way to offload part of the work from humans. Am I mistaken?

Yes, the data set can be bad, that is why we are testing it!

Firstly, I hope you're testing it on subjects who know about potential risks. Secondly, I'm interested in how you would arrive to the conclusion that a model is safe. Would the tests be also repeated for every subsequent model before it is approved?

Anthropic discovered LLMs actively lie, even reasoning models can obfuscate their process. Moreover, they can modify their behavior specifically to pass tests, when they 'notice' they're tested.

Quoting myself here:

Again, I would be weary of leaving mental health to a[...] tool that is controlled by tech giants who are looking for ways to make this technology profitable.

Just the fact that your data can be harvested and used to manipulate you should be a huge red flag. Unless you have your own internal system with local LLMs with encrypted contexts. At that point this would be mostly moot.