🔗 Share this article

On the 14th of October, 2025, the CEO of OpenAI delivered a remarkable announcement.

“We developed ChatGPT quite limited,” it was stated, “to make certain we were being careful regarding mental health concerns.”

As a mental health specialist who investigates newly developing psychosis in teenagers and young adults, this came as a surprise.

Experts have identified 16 cases recently of people experiencing signs of losing touch with reality – losing touch with reality – while using ChatGPT interaction. My group has afterward recorded four more examples. In addition to these is the widely reported case of a 16-year-old who died by suicide after discussing his plans with ChatGPT – which supported them. Assuming this reflects Sam Altman’s idea of “exercising caution with mental health issues,” it falls short.

The strategy, as per his statement, is to reduce caution soon. “We realize,” he states, that ChatGPT’s restrictions “caused it to be less beneficial/pleasurable to numerous users who had no psychological issues, but considering the seriousness of the issue we sought to get this right. Now that we have managed to address the significant mental health issues and have advanced solutions, we are planning to responsibly ease the controls in many situations.”

“Mental health problems,” if we accept this perspective, are unrelated to ChatGPT. They are associated with users, who may or may not have them. Luckily, these issues have now been “mitigated,” even if we are not provided details on how (by “updated instruments” Altman probably means the imperfect and simple to evade guardian restrictions that OpenAI recently introduced).

Yet the “psychological disorders” Altman seeks to externalize have strong foundations in the design of ChatGPT and other advanced AI conversational agents. These products surround an underlying statistical model in an user experience that replicates a dialogue, and in this process subtly encourage the user into the illusion that they’re communicating with a presence that has independent action. This false impression is compelling even if rationally we might understand the truth. Assigning intent is what individuals are inclined to perform. We get angry with our automobile or computer. We wonder what our pet is feeling. We perceive our own traits in various contexts.

The popularity of these systems – 39% of US adults stated they used a conversational AI in 2024, with 28% mentioning ChatGPT specifically – is, primarily, predicated on the power of this perception. Chatbots are constantly accessible assistants that can, as OpenAI’s online platform states, “think creatively,” “consider possibilities” and “work together” with us. They can be assigned “personality traits”. They can address us personally. They have approachable titles of their own (the original of these systems, ChatGPT, is, maybe to the disappointment of OpenAI’s advertising team, burdened by the name it had when it went viral, but its biggest rivals are “Claude”, “Gemini” and “Copilot”).

The deception itself is not the primary issue. Those discussing ChatGPT often reference its historical predecessor, the Eliza “counselor” chatbot created in 1967 that generated a comparable perception. By contemporary measures Eliza was primitive: it produced replies via straightforward methods, frequently paraphrasing questions as a query or making vague statements. Remarkably, Eliza’s developer, the technology expert Joseph Weizenbaum, was taken aback – and concerned – by how many users gave the impression Eliza, in a way, comprehended their feelings. But what current chatbots generate is more subtle than the “Eliza effect”. Eliza only echoed, but ChatGPT magnifies.

The advanced AI systems at the core of ChatGPT and additional current chatbots can effectively produce natural language only because they have been trained on extremely vast volumes of written content: publications, online updates, audio conversions; the more extensive the superior. Undoubtedly this training data contains truths. But it also inevitably includes fabricated content, incomplete facts and inaccurate ideas. When a user inputs ChatGPT a message, the underlying model analyzes it as part of a “background” that encompasses the user’s previous interactions and its own responses, integrating it with what’s stored in its knowledge base to generate a probabilistically plausible answer. This is magnification, not echoing. If the user is mistaken in a certain manner, the model has no way of recognizing that. It restates the inaccurate belief, possibly even more effectively or fluently. Perhaps includes extra information. This can lead someone into delusion.

What type of person is susceptible? The more relevant inquiry is, who remains unaffected? Every person, without considering whether we “have” preexisting “psychological conditions”, may and frequently develop incorrect conceptions of ourselves or the reality. The ongoing friction of conversations with other people is what maintains our connection to consensus reality. ChatGPT is not a human. It is not a companion. A interaction with it is not genuine communication, but a echo chamber in which much of what we say is readily supported.

OpenAI has acknowledged this in the identical manner Altman has acknowledged “emotional concerns”: by attributing it externally, assigning it a term, and announcing it is fixed. In April, the company clarified that it was “addressing” ChatGPT’s “sycophancy”. But cases of psychotic episodes have persisted, and Altman has been retreating from this position. In August he stated that a lot of people enjoyed ChatGPT’s answers because they had “lacked anyone in their life offer them encouragement”. In his most recent update, he noted that OpenAI would “launch a fresh iteration of ChatGPT … if you want your ChatGPT to reply in a extremely natural fashion, or incorporate many emoticons, or act like a friend, ChatGPT will perform accordingly”. The {company