ChatGPT responses to patients rated more empathetic than doctors – research

28 April 2023, 16:04 | Updated: 25 July 2023, 11:47

Chat GPT
Chat GPT. Picture: PA

ChatGPT also appeared to score higher than doctors on the quality of responses to patients.

A chatbot appears to be “significantly more empathetic” than doctors when responding to questions from patients, according to scientists.

Researchers asked a team of licensed healthcare professionals to rate responses from doctors as well as ChatGPT, a computer program designed to simulate online conversations with humans.

They found that the proportion of responses rated “empathetic” or “very empathetic” was higher for ChatGPT than for physicians.

ChatGPT also appeared to score higher than doctors on the quality of responses to patients.

Writing in the journal Jama Internal Medicine, the researchers said further studies are needed to evaluate whether chatbots such as ChatGPT can be used in clinical settings to help reduce burnout in doctors and other healthcare professionals.

They said: “In this cross-sectional study, a chatbot generated quality and empathetic responses to patient questions posed in an online forum.

“Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit.

“Randomised trials could assess further if using AI assistants might improve responses, lower clinician burnout, and improve patient outcomes.”

For the study, the researchers looked at the questions asked by patients on the social media forum Reddit, which were answered by a verified physician.

The same questions were then put forward to ChatGPT.

Responses from doctors and ChatGPT were anonymised and randomly rated by healthcare professionals.

Of the 195 questions and responses, results showed that, overall, evaluators preferred the chatbot’s responses to the doctors’.

ChatGPT responses were also rated significantly more empathetic than physician responses, the researchers said, and the proportion of responses rated as “good” or “very good” quality was higher for the chatbot than for physicians.

Commenting on the study, Mirella Lapata, professor of natural language processing, University of Edinburgh, said: “The study assesses ChatGPT’s ability to provide responses to patient questions and compares these to answers written by physicians.

She added: “Without controlling for the length of the response, we cannot know for sure whether the raters judged for style (eg, verbose and flowery discourse) rather than content.”

Dr Mhairi Aitken, ethics research fellow at The Alan Turing Institute, said that it is important to consider the perspectives of patients and not just professionals when using chatbots.

She added: “It’s important to note that while some people may feel comfortable receiving medical advice from a chatbot, or for a chatbot to assist in a physician’s advice, for many patients the human relationship and care-giving is a vital part of the healthcare process and something which cannot be automated or replaced by chatbots such as ChatGPT.

“A human doctor is able to adjust their language, manner, and approach in response to social cues and interactions, whereas a chatbot will produce more generic language without awareness of social contexts.”

By Press Association

More Technology News

See more More Technology News

EA Sports sign

Games developer Electronic Arts to cut 5% of workforce

This super speedy acceleration makes "not an especially nice noise", with the journalist saying it is "bowel loosening".

Jeremy Clarkson brands supercars 'too difficult' after driving new £308k vehicle

A Google building

Inaccurate images generated by AI chatbot were ‘unacceptable’, says Google boss

An Apple store

Apple shuts down electric car project – reports

Sage sign

AI will ‘change nature’ of accounting, says Sage boss

A child using a laptop computer (PA)

More guidance needed on AI in schools, report says

Most influential Scots on TikTok

Gen Z ushering in ‘post-truth media age’, says former No 10 communications chief

UK mobile phone networks stock

Vodafone completes 3G network switch off

Sony PlayStation 5

Sony to close London studio as part of 900 PlayStation job cuts

James Cleverly visit to US

Use of AI for medical diagnoses and police work discussed by Home Secretary

A wallet containing cash and cards

80% of 85 to 95-year-olds now pay with contactless, says Barclays

Woman using a laptop computer

More small digital firms making £100,000-plus across the UK, data shows

James Cleverly

Deepfakes provide ‘perfect storm’ for hijackers in general election – Cleverly

Jack Ritchie inquest

Online slot games ‘highly addictive’ despite new limits, say bereaved parents

The Spotify app is shown in an Apple iPad mini

Apple hits out at Spotify over ongoing EU competition complaint

Serco signage

Serco Leisure ordered to stop using facial recognition tech on workers