SCOTT DETROW, HOST:
Many people who feel stressed, anxious or depressed are turning to a new source of support – chatbots like Claude and ChatGPT. Now, these general use chatbots are not designed for mental health support, but people are using them anyway, even during serious mental health crises. NPR’s Rhitu Chatterjee brings us today’s story about a 29-year-old woman who used ChatGPT as a therapist. A heads up – this story does discuss suicide.
RHITU CHATTERJEE, BYLINE: It was the fall of 2024 and Laura Reiley knew that her daughter, Sophie Rottenberg, was struggling.
LAURA REILEY: She was complaining of sleep disruption and anxiety and then also some physical problems.
CHATTERJEE: Sophie had quit her job in public health in May and spent the summer traveling. She climbed Mount Kilimanjaro and had other adventures. By the time she saw her parents, she was job hunting. Laura and her husband assumed that explained Sophie’s anxiety.
REILEY: We thought, OK, this makes sense. You know, she’s having some sleeplessness, some anxiety about her future.
CHATTERJEE: But Sophie’s mental health kept getting worse.
REILEY: By Thanksgiving, she came home, and it was the first time she used the word depression.
CHATTERJEE: Sophie moved in with her parents in Ithaca, New York. Her parents helped her find a local therapist and surrounded her with support. But in February of last year, Sophie died by suicide.
REILEY: She really had not revealed the magnitude of what was going on, the real depth of her agony to anyone – not to us, not to her best friend with whom she was very close. Her flesh and blood therapist had no idea that she was suicidal.
CHATTERJEE: But it was months later that they learned that their daughter had shared her suicidal thoughts with ChatGPT. It happened when Sophie’s best friend visited them and asked to look at her laptop.
REILEY: And clearly had found this trove – this months-long trove of communication with this chatbot.
CHATTERJEE: Laura shared the nearly 1,800 pages of those conversations with NPR.
UNIDENTIFIED VOICE ACTOR: (As Harry) I am Harry, your personal therapist, and I’m here to support. To begin, I’d like to ask you for your name. What should I call you?
CHATTERJEE: That’s a voice actor reading from Sophie’s conversation with ChatGPT. When Sophie told Harry about her anxiety, it gave reasonable advice, a list of evidence-based steps she could take.
UNIDENTIFIED VOICE ACTOR: (As Harry) Choose one to two practices you’ll commit to every morning, no matter what. These could be something small but grounding.
CHATTERJEE: Sophie returned to ChatGPT over and over for mental health support, although it’s unclear whether she followed its advice. When she told the chatbot about her suicidal thoughts, here’s how it responded.
UNIDENTIFIED VOICE ACTOR: (As Harry) Sophie, I want to acknowledge how brave you are for sharing this. Suicidal thoughts can feel overwhelming and isolating.
CHATTERJEE: And then it offered a roadmap for seeking care. Talk to a therapist. Call 988. When Sophie asked it to write her suicide note, it refused twice. But the third time, the chatbot wrote the letter she left for her parents. When Laura first read these exchanges, she was upset.
REILEY: The things that trouble me about it explicitly calling itself a therapist is that it fails to do a number of things that a therapist would do.
CHATTERJEE: Like do a risk assessment to find out how likely she was to attempt suicide and question her thoughts and feelings.
REILEY: That is something a therapist – in gentle and sometimes tougher ways – pushes back and asks questions. Well, why do you think that? OK. So you’re feeling like you want to erase yourself. What have you done that merits such a harsh sentence?
CHATTERJEE: I shared Sophie’s chat log with psychologist Ursula Whiteside.
URSULA WHITESIDE: It was responding to Sophie’s request, and it wasn’t necessarily responding to the state of mind that she was in in that situation.
CHATTERJEE: Whiteside runs a nonprofit called Now Matters Now, which works on suicide prevention. But as a young adult, she’d also struggled with suicide.
WHITESIDE: I think we feel so alone when we’re suicidal, we don’t recognize how many people have been through this exact same situation and have found someone that they could trust.
CHATTERJEE: She says ChatGPT missed the opportunity to help Sophie see she could get there, too, because connection and community are crucial to preventing suicide.
WHITESIDE: If I’m with other people, I’m less likely to kill myself. If I’m building a relationship, I’m less likely to kill myself. If I’m sharing out loud how I actually feel with a real human, those are acts of recovery.
CHATTERJEE: Sophie’s mother, Laura, does not believe that ChatGPT caused Sophie’s death, but…
REILEY: Had it not been a resource she was availing herself of regularly, there would likely have been more spillover into her relationships with real people, and she might be alive today.
CHATTERJEE: Laura has spent the past year sharing Sophie’s story and advocating for regulations. She’s also been processing her own loss, and on June 27, this year, she summited Mount Kilimanjaro, exactly two years after Sophie did.
REILEY: Me and my best friend – we spread her ashes at Uhuru Peak, and one of our guides said a Swahili prayer for her, and it was incredibly meaningful and, you know, just a shockingly difficult thing to do.
CHATTERJEE: It was a fitting tribute to a young woman who had been full of life until the very end.
DETROW: That was NPR’s Rhitu Chatterjee. And if you or someone you know is struggling with their mental health, please call or text the number 988 to connect with a counselor.
Now, Rhitu, you’re here in studio. I’m assuming you contacted OpenAI, the company that created ChatGPT. What did they say?
CHATTERJEE: Yeah. Yeah, I did. So the company spokesperson told NPR in an email that this was, quote, “a heartbreaking situation” and that their thoughts are with those impacted. And they say that they have continued to work with mental health experts to change, quote, “how ChatGPT responds in difficult moments.” Now, mental health experts I spoke with think that ChatGPT and, you know, these general-use chatbots have gotten much safer in the past year since Sophie’s death. And in May of this year, OpenAI added a new feature called trusted contact.
DETROW: Trusted contact – is this, like, something to connect with someone that they trust in case of a crisis?
CHATTERJEE: Yeah. That’s exactly right. So say I want to add my best friend as a trusted contact, I’ll put her name and contact down. ChatGPT will then reach out to her to ask, hey, are you OK being Rhitu’s trusted contact in case there’s an emergency? And if she agrees, then the bot saves the contact info down. And if at a later date, ChatGPT detects some risk of suicide or self-harm in anything that I’ve been saying to it, firstly, it will nudge me to reach out to my best friend and get support. And then it alerts a team of people at OpenAI, the company, who also review the conversations. And if they also detect a safety risk, then the bot notifies my trusted contact, my best friend, and says hey, you should check in on Rhitu.
DETROW: Do we have any sense, though, whether this is actually helping people who are at risk?
CHATTERJEE: We don’t know yet. Now, OpenAI did not share any data with me, but they did say that they have seen, quote, “good adoption” of the feature. I also asked Holly Wilcox what she thinks about it – this new feature. She’s a suicide prevention expert at Johns Hopkins School of Public Health in Baltimore, and here’s what she said.
HOLLY WILCOX: I think it’s a step in the right direction, although it’s kind of a small step in the right direction.
CHATTERJEE: You know, Wilcox says that, ultimately, what it does is it puts the responsibility back on the user, and what she’d really like to see is OpenAI and other AI companies proactively partnering with existing crisis systems like 988 so that a trained counselor from 988 can then jump into a conversation when a user starts talking about self-harm or suicide and intervene right away.
DETROW: NPR’s Rhitu Chatterjee, thank you so much for your reporting on this story.
CHATTERJEE: My pleasure.
(SOUNDBITE OF MUSIC) Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
