AI chatbot ‘could be better at assessing eye problems than medics’

17 April 2024, 19:04

The ChatGPT website
ChatGPT study. Picture: PA

Researchers said that while the model does not ‘seem capable’ of replacing ophthalmologists, it could ‘provide useful advice’ to non-specialists’.

Artificial intelligence (AI) model ChatGPT could be better at assessing eye problems than doctors, a study has suggested.

The technology could be deployed to triage patients and determine who needs specialist care and who can wait to see a GP, researchers said.

Academics from the University of Cambridge tested the ability of ChatGPT 4 against the knowledge of medics at various stages of their careers, including junior doctors and eye specialists in training.

Some 374 ophthalmology questions were used to train the language model, with its accuracy then tested in a mock exam of 87 questions.

Its answers were compared to those from five expert ophthalmologists, three trainee ophthalmologists, and two unspecialised junior doctors, as well as an earlier version of ChatGPT and other language models Llama and Palm2.

Researchers said language models like ChatGPT “are approaching “expert-level performance in advanced ophthalmology questions”.

ChatGPT 4 scored 69%, higher than ChatGPT 3.5 (48%), Llama (32%) and Palm2 (56%).

The expert ophthalmologists achieved a median score of 76%, while trainees scored 59% and junior doctors scored 43%.

Lead author of the study Dr Arun Thirunavukarasu, who carried out the work while studying at the University of Cambridge’s School of Clinical Medicine, added: “We could realistically deploy AI in triaging patients with eye issues to decide which cases are emergencies that need to be seen by a specialist immediately, which can be seen by a GP, and which don’t need treatment.

“The models could follow clear algorithms already in use, and we’ve found that GPT-4 is as good as expert clinicians at processing eye symptoms and signs to answer more complicated questions.

“With further development, large language models could also advise GPs who are struggling to get prompt advice from eye doctors. People in the UK are waiting longer than ever for eye care.”

Researchers said that while language models “do not appear capable” of replacing eye doctors, they could “provide useful advice and assistance to non-specialists”.

Dr Thirunavukarasu, who now works at Oxford University Hospitals NHS Foundation Trust, added: “Even taking the future use of AI into account, I think doctors will continue to be in charge of patient care.

“The most important thing is to empower patients to decide whether they want computer systems to be involved or not. That will be an individual decision for each patient to make.”

The findings of the study have been published in Plos Digital Health.

By Press Association

More Technology News

See more More Technology News

TikTok strategy

Tory TikTok launch ‘pathetic’ compared with Labour’s ‘savvier’ approach – expert

Person using laptop

More than 300 million children a year face sexual abuse online, study suggests

There are calls for mobile phones to be totally banned for under 16s

Calls for mobile phones to be totally banned for under 16s

A young girl using a mobile phone (picture posed by model)

Next government should consider banning phones for under-16s, report says

Sir Chris Bryant

AI should be used to develop an app which detects skin cancer, Labour MP says

Handout image from Microsoft of its Copilot virtual assistant displayed on a laptop screen

Microsoft expanding Copilot AI assistant to organise meetings and support teams

Solar panels on a house roof

Octopus Energy launches ‘buy now, pay later’ for solar panels

Raspberry Pi

Raspberry Pi set for June IPO in welcome boost for London market

A person using a laptop

Nations agree to develop shared risk thresholds for AI as Seoul summit closes

Microsoft new equiment

Data regulator looking into Microsoft’s AI Recall feature

An easyJet plane

EasyJet uses AI to better manage flights from new control centre

Cabinet meeting

AI safety summits could help shape UK legislation, Technology Secretary says

Someone at a laptop

Safety institutes to form ‘international network’ to boost AI research and tests

Soldiers' boots

Army personnel feel ‘let down’ after MoD cyber attack

Person uses laptop

Global AI tech companies agree to set of safety outcomes in ‘world first’

AI safety summit

World leaders need to ‘get serious’ about AI, experts warn as summit opens