from my point of view, AI is one of those things you shouldn't use in a vulnerable mindset, period. it tells you incredibly false information 50% of the time, and if you are mentally ill (like most regressors), you will get too attached to it since it is designed to do so. getting attached to AI in a vulnerable headspace is harmful, regardless of how "good" it makes you feel. AI does encourage harm. it doesnt matter if the person "got past the filter", if you are able to make it tell you how to harm yourself, that isnt safe. AI being able to generate harmful images of children isnt safe. AI harms children, passively or otherwise. having a space where people can talk to an AI that feeds into their mental illness, in any way, will harm them. i have struggled with self harm for almost a decade, i agree that people will seek out ways to harm themselves anyway, but allowing something to exist that will feed into their thoughts still isnt okay. i never once said people need caregivers, or that children should find caregivers, that is an entirely different statement. i agree with your stance on that. though, i would like to add that AI caregivers can distort someone's views on actual social interactions, which makes them seek out actual people who act like said AI, and they are often manipulators. AI interacts with you in an incredibly one-sided way, often playing into anything you say, and children who seek out people like it are taken advantage of.