OpenAI has said it is working on upgrades that will help improve ChatGPT’s ability to detect signs of mental or emotional distress among the users of the artificial intelligence chatbot.
These changes will let the AI chatbot “respond appropriately and point people to evidence-based resources when needed”, the Microsoft-backed AI startup said in a blog post on Monday. OpenAI is also working with a wide range of stakeholders including physicians, clinicians, human-computer-interaction researchers, mental health advisory groups, and youth development experts to improve ChatGPT’s responses in such cases.
The company further said that ChatGPT will be tweaked so that its AI-generated responses are less decisive in “high-stakes situations”. For example, when a user asks a question like “Should I break up with my boyfriend?”, the AI chatbot will walk the user through the decision by asking follow-up questions, weighing pros and cons, etc, instead of giving a direct answer.
This behavioural update to ChatGPT for high-stakes personal decisions will be rolling out soon, it said.
OpenAI’s new efforts to promote the healthy use of ChatGPT come against the backdrop of an alarming number of users turning to AI chatbots for therapy and professional advice. This emerging trend has sparked concerns among mental health professionals who have cautioned that AI chatbots may not be equipped to offer appropriate guidance and may end up having an amplifying effect on some users’ delusions as they are designed to generate outputs that please users.
In April this year, OpenAI rolled back an update that made ChatGPT too agreeable and sycophantic. Acknowledging past failures, OpenAI said on Monday, “We rolled it back, changed how we use feedback, and are improving how we measure real-world usefulness over the long term, not just whether you liked the answer in the moment.”
“We also know that AI can feel more responsive and personal than prior technologies, especially for vulnerable individuals experiencing mental or emotional distress,” it added.
Story continues below this ad
The company is expected to launch its next major large language model, GPT-5, this week. Its ChatGPT platform is reportedly on track to touch 700 million weekly active users.
Reminders on ChatGPT
As part of its efforts to promote the healthy use of ChatGPT, OpenAI also said it is bringing reminders to the platform. These reminders will be shown to users who have been chatting with the AI chatbot for a while.
“Starting today, you’ll see gentle reminders during long sessions to encourage breaks. We’ll keep tuning when and how they show up so they feel natural and helpful,” OpenAI said.
The reminders will appear as pop-up messages with a rounded white box on a soft blue gradient background. It will read, “Just checking in — You’ve been chatting a while — is this a good time for a break?” with two buttons labelled “Keep chatting” and “This was helpful”.
Story continues below this ad
Other popular social media platforms such as YouTube and Instagram have launched similar features in recent years. Earlier this year, Character AI said it will send parents and guardians a weekly summary over email to keep them informed of an underage user’s activities on the platform.
These measures were announced after the Google-backed startup was accused of exposing underage users to “hyper-sexualised content” and promoting self-harm through their role-playing AI chatbots. These allegations were detailed in two lawsuits filed by parents of children who had signed up for Character AI.