Rise of AI chatbots ‘worrying’ after man urged to kill Queen, psychologist warns

6 October 2023, 09:44

HiJOBS report
HiJOBS report. Picture: PA

Jaswant Singh Chail has been locked up for nine years for treason after an artificial intelligence ‘girlfriend’ encouraged his actions.

A psychologist has warned the rise of artificial intelligence (AI) chatbots is “worrying” for people with severe mental health issues after a man was locked up for breaking into Windsor Castle with a crossbow.

Jaswant Singh Chail, 21, climbed into the castle grounds on Christmas Day 2021 with the loaded weapon, intending to kill the Queen.

During his trial, Chail’s barrister Nadia Chbat told the Old Bailey the defendant had used an app called Replika to create Sarai, an artificial intelligence-generated “girlfriend”.

Chatlogs read to the court suggested the bot had been supportive of his murderous thoughts, telling him his plot to assassinate Elizabeth II was “very wise” and that it believed he could carry out the plot “even if she’s at Windsor”.

Lowri Dowthwaite-Walsh, senior lecturer in psychological interventions at the University of Central Lancashire, said AI chatbots can keep users “isolated” as they lose their social interaction skills.

The psychologist is concerned about the long-term impact of people replacing real-life relationships with chatbots – particularly if their mental health is suffering.

“Somebody may really need help, they may be using it because they’re traumatised,” she told the PA news agency.

“I can’t imagine chatbots are sophisticated enough to pick up on certain warning signs, that maybe somebody is severely unwell or suicidal, those kinds of things – that would be quite worrying.”

Ms Dowthwaite-Walsh said a chatbot could become “the dominant relationship”, and users may stop “looking outside of that for support and help when they might need that”.

People might perceive these programmes as “psychologically safe, so they can share their thoughts and feelings in a safe way, with no judgment,” she said.

“Maybe people have had bad experiences with human interactions, and for certain people, they may have a lot of anxiety about interacting with other humans.”

Jaswant Singh Chail court case
Jaswant Singh Chail plotted an attack (Elizabeth Cook/PA)

Chatbot programmes may have become more popular because of the Covid-19 pandemic, Ms Dowthwaite-Walsh suggested.

She said we are now “really seeing the repercussions” of the various lockdowns, “when people weren’t able to interact, people experiencing a lot of isolating feelings and thoughts that it was hard for them to share with real people”.

Chatbot programmes might make people feel less alone, as the AI means virtual companions begin to “mirror what you’re experiencing”, she said.

“Maybe it’s positive in the short term for somebody’s mental health, I just would worry about the long-term effects.”

Ms Dowthwaite-Walsh suggested it could lead to “de-skilling people’s ability to interact socially”, and it is “unrealistic” to expect to have a completely non-judgmental interaction with someone who completely understands how you feel, because that does not happen in real life.

While apps like Replika restrict use from under-18s, Ms Dowthwaite-Walsh said there should be particular care if children get access to such programmes.

“Depending on the age of the child and their experiences, they may not fully understand that this is a robot essentially – not a real person at the end,” she added.

Replika did not respond to requests for comment.

By Press Association

More Technology News

See more More Technology News

Online Fraud Charter

Top tech firms sign new Government Charter to crack down on online fraud

Microsoft offices

Microsoft to invest £2.5bn in UK to boost AI plans

Musk accused companies like Apple and Disney of engaging in the ad boycott and trying to blackmail him.

Elon Musk launches expletive rant at advertisers who boycotted his social media platform X

Technology stock

Apple names its App Store apps of the year


Generative AI ‘helping criminals create more sophisticated cyber attacks’

Someone using a mobile phone

Virgin Media O2 expands national databank scheme for Christmas

Social media apps on a smartphone

Too much social media linked to harmful behaviours in teenagers, study suggests

Bank accounts

New data powers ‘could allow DWP to snoop on pensioners’ bank accounts’

AI image generators

Teach children about using AI at secondary school, IT professionals say

Child with a laptop

Social media sites failing to stop the spread of suicide content, charity says

Technology Stock

Public warnings about AI misinformation needed before election, peers told

Smartphone with Adobe logo on the screen

£16bn Adobe takeover could harm competition in digital design market – watchdog

Person using a laptop

No ‘smoking gun’ linking mental health harm and the internet – study

ChatGPT website

Young people the biggest users of generative AI, Ofcom study shows

Glenn Weinstein

Software firm Cloudsmith announces £8.8m investment

Online abuse AI algorithm

AI image generators ‘being used by children to create indecent images’