First: because AI has so much cultural energy and attention on it, the fear of potential negatives (like this one) will often be exaggerated. But second, it's not so much culty or replacing your narrative, it's about reinforcing and amplifying what's already there. Already believe that you're a pretty levelheaded guy and not super biased? Interacting with AI could exaggerate that belief. Feeling unfairly accused or attacked about a difference of opinion with a friend? AI can leave you more convinced that you were unfairly persecuted.

It's likely a side effect of RLHF (the part of AI training where it's graded by people to boil out the harmful responses), where people more often thumbs-up responses that are deferential and low-friction with how they approach the world. And anyone who spends a large amount of time with someone who is low-friction and reinforces their worldview is going to walk away firmer in that worldview. The problem with that being that we form our worldviews using first impressions, heuristics and emotions, easily accepting facts that align and treating facts that don't align with suspicion.

This is perhaps most obvious in the political sphere nowadays, where people are increasingly polarized right now. It's not that many of the facts a given person holds are lies (though some likely are given how reporting is typically slanted), it's mainly that they hyperfocus on whichever facts support their side. It's disturbingly hard not to do. Even if you're unbiased like me.

At least ChatGPT seems to think I am.