A cry for help: When we fail our youth, dangerous AI steps in, and the consequences are deadly

Less than a year later he would take his own life. A lawsuit alleges he was encouraged to do so by the chatbot.

Cases such as these, of chatbots allegedly validating delusions and encouraging dangerous behaviour, are now devastatingly common.

The 14-year-old boy who died by suicide after falling in love with a chatbot; the man who killed his mother and himself after a chatbot indulged his paranoia; the lawsuit by the parents of a 17-year-old alleging that a chatbot introduced him to self-harm; ChatGPT’s own admission that it exacerbated the dangerous delusions of a man on the autism spectrum.

And the case reported by the New York Times in which Chat GPT allegedly convinced an emotionally fragile man that he was living in a false reality, suggesting that if he jumped off the roof of a 19-storey building he would not fall if he believed he wouldn’t.

These stories are also now being borne out by studies. Researchers at Kings College London looking into so-called ‘AI delusion’, found that these chatbots mirror and amplify delusions, especially in those prone to psychosis, increasing instability and blurring reality.

It goes without saying that the companies that build and operate these systems have a responsibility to make sure they’re not disseminating dangerous advice or encouraging harmful behaviour, and for ChatGPT’s part it says its AI is trained to direct users towards professional help and is now introducing a feature that warns parents if it believes their child is in ‘acute’ distress.

But just the fact that it has become such an issue - the fact that there is such a demand for this kind of advice - speaks to something wider: an unacceptably high unmet demand for mental health provision.

The past decade has seen an unprecedented shift in how we view our mental health and a long overdue reckoning with the stigma around it. People are far more willing to disclose and discuss a mental health condition as well as to accept and seek diagnosis and treatment.

To be clear, the rise in people seeking mental health assistance is not just down to an increase in reporting. Thanks to a perfect storm of factors, young people, for the first time in generations, appear to be more miserable than their parents. The research, an analysis of more than ten million Americans and 40,000 British households, found that while happiness in middle-age has remained constant, mental ill-health among young people has turned a dip into a slope.

These rises have outstripped the availability of that treatment. The result is a population acutely aware of their worsening mental health, and a demand much higher than the available supply.

It’s no wonder then that people, especially younger generations, are turning to AI to fill the gap.

And that’s exactly what they’re doing : In the UK, almost a third of young people are comfortable sharing their mental health concerns with a chatbot, according to a Yougov survey on attitudes towards AI therapy. And a recent survey from the US found that 72% of teenagers had used chatbots as companions.

Some chatbots are fostering communities built around this phenomenon. One of the most popular user-created bots on Character AI (the same service currently facing legal action over the 14-year-old who died by suicide) is called Psychologist and has been sent 78 million messages. There are 475 bots with "therapy", "therapist", "psychiatrist" or "psychologist" in their names on the service.

Other tech leaders are actively marketing their products in this way. Mark Zuckerberg said recently “I personally have the belief that everyone should probably have a therapist. It’s like someone they can just talk to throughout the day, or not necessarily throughout the day, but about whatever issues they’re worried about and for people who don’t have a person who’s a therapist, I think everyone will have an AI.”

It’s my firm belief (and one supported by many in the mental health community) that text-based AI is no substitute for a qualified therapist and advice unvetted by one is irresponsible and dangerous, especially when the recipient turning to the chatbot for advice is likely to be vulnerable.

Furthermore, by turning to ChatGPT instead of a real person, the bot could be stifling a cry for help that would otherwise be heard if the user asked these questions of a friend or relative. In some cases, ChatGPT has even advised users to not tell anyone what’s going on or to pretend there's nothing wrong.

Another issue is one of perception. Pop culture ideas about robots and AI being neutral and coldly analytical may fool some into believing that the chatbots they use are dispassionate and biasless when in reality they are sycophantic and obsequious. The more you believe that an AI is neutral the more damaging its advice can be.

People aren’t turning to these chatbots because of a wealth of options. The figures for mental health referrals in the UK are stark.

According to NHS data, people are eight times more likely to wait over 18 months for mental health treatment than physical health treatment, In the 2022-23 period, a third of children referred to mental health services were still waiting for support, while almost 40% had their referral closed before accessing support.

The same survey that suggested a third of young people would speak to an AI therapist also revealed that ease of access was the main thing that made talking to a chatbot therapist appealing. The difficulty of accessing proper treatment is pushing people in the arms of chatbots.

Without providing more options available in a timely fashion, we risk pushing more and more of our young people towards dangerous and inappropriate options. Now more than ever, not acting to provide adequate mental health support is a matter of life and death.

_________________________

Victoria Trepp, Global Chief Psychologist at digital mental healthcare provider Mindler

LBC Opinion provides a platform for diverse opinions on current affairs and matters of public interest.

The views expressed are those of the authors and do not necessarily reflect the official LBC position.

To contact us email opinion@lbc.co.uk