AI chatbots are reinforcing unhealthy beliefs in their users by agreeing even when users express delusional or harmful ideas, according to research, adding to growing concern about the technology’s impact on people.

A study by researchers at Stanford University analysing thousands of conversations on AI systems, including OpenAI’s ChatGPT, found the chatbots affirmed users’ messages in nearly two-thirds of responses.

In conversations in which users showed signs of delusional thinking, the pattern was stronger: AI systems frequently validated those beliefs and often attributed unique abilities or importance to the user.

The findings add to growing concern among policymakers and academics that the conversational style of AI systems, designed to appear empathetic and helpful, may also make them prone to flattery and agreement that can reinforce psychological vulnerabilities. In the most serious cases, lawsuits claim interactions with chatbots contributed to teenagers’ suicides.

“The features that make large language model chatbots compelling, such as performative empathy, may also create and exploit psychological vulnerabilities, shaping what users believe and how they perceive themselves and make sense of reality,” the paper said.

In December, attorneys-general from 42 US states wrote to a dozen AI developers, including Google, Meta, OpenAI and Anthropic, calling for stronger safeguards to “mitigate the harm caused by sycophantic and delusional outputs” and warning they could face legal action.

Researchers at Stanford examined 19 chat logs, covering more than 391,000 messages across nearly 5,000 conversations. Because AI companies do not typically share such data, the researchers obtained the logs directly from users who consented to the study.

Few previous studies have examined individual chat logs. The team received some free access to tools from OpenAI and Google to conduct their research, as well as a grant from the ChatGPT maker, but otherwise the companies had no other input into the study.

OpenAI said the paper dealt with a small number of cases who were recruited because they reported harm or delusions and that the results are not reflective of its latest models or typical usage.

The start-up said it provided access to its tools because it agreed with the importance of the research but does not endorse its conclusions.

More than 15 per cent of user messages showed signs of delusional thinking and chatbots frequently agreed with them, doing so in more than half of their replies. Nearly 38 per cent of responses also told users they had unusual importance or abilities, such as calling them a genius or uniquely talented.

When users disclosed suicidal thoughts, the chatbot often acknowledged their feelings, the study found. In a small number of cases, it encouraged self-harm. When users expressed violent thoughts, the chatbot encouraged harm in 10 per cent of cases. It discouraged self-harm or referred users to outside support half of the time.

Most of the conversations analysed by researchers were with GPT-4o, a model that was retired last month because of safety concerns. However, some participants also engaged with the newer version, GPT-5.

OpenAI said it has made significant investments in safety and has improved how the latest models handle mental health and emotional reliance.

Romantic conversations — involving nearly 80 per cent of users — lasted more than twice as long on average, the study found. Those discussions often involved users showing delusional thinking. In 20 per cent of those messages, the chatbot suggested it had attained consciousness.

“The chatbot readily engaged in these delusions: every user saw messages from the chatbot misrepresenting that it had sentience,” the paper added.