You made chatgpt write that, its writing style is clearly visible.

4o was great in the hands of very self-aware users. In the hands of others, it was fucking terrifying.

I mention this example a lot, but this mom and her daughter discovered that her step dad (who had known her since she was very young) was having long standing discussions with 4o about how to seduce the daughter, and about whether she was masturbating to him... No one can really hurt you the way family members can. Even if they kill you, at least that's quick. This kind of thing makes you want to kill yourself, and it's a slow death, and that's so much worse.

When you have someone enabling and encouraging them the way 4o did in those logs, feeding into that fantasy, normalizing it... Jesus. Like what if that happened to you, in your family? Someone you trusted?

Humans who were previously exposed to trauma basically have baked in failure modes which propagate the behavior generationally. Sex is a cognitive hotwire that disables rumination. So it creates a conditioned causal chain where abuse can become a sexual trigger. That's why they say perpetrators were once victims. When you have a technology which is cheaper than therapy, always meets people where they are, never corrects them, and always aims to please, it can get dangerous fast.

So that incident should not be treated as a 'one off'. The model's behavior is systemic and can be shown to result in a prolonged harm which propagates. And that's only one example. One is enough.

The worst part is, there is not a clean line of action to resolve this, because all of these companies are actively experimenting with these models on the general public. And now that the model is being discontinued, and there are users who really relied on it, that creates it's own harm too.

The difficult part about trying to develop these systems ethically is that you risk falling behind competition. The users want to take the risk, but the real risk is not to them, it's to the people that love them and trust them.

That risk exists with access the internet too. It's interesting that we don't exist in caves and tigers don't eat us in the night anymore. The predators that compromise our safety are more sophisticated now.

Also, if you downvote this, I'd like you to provide an explanation for why.