AI being used to create child abuse imagery, watchdog warns

25 October 2023, 00:04

Online abuse
Online abuse. Picture: PA

The Internet Watch Foundation has warned a ‘nightmare’ scenario is developing around AI-generated abuse imagery online.

Thousands of AI-generated images depicting real victims of child sexual abuse threaten to “overwhelm” the internet, a watchdog has warned.

The Internet Watch Foundation (IWF), the UK organisation responsible for detecting and removing child sexual abuse imagery from the internet, said its “worst nightmares” have come true.

The IWF said criminals were now using the faces and bodies of real children who have appeared in confirmed abuse imagery to create new images of sexual abuse through artificial intelligence technology.

The data published by the organisation said the most convincing imagery would be difficult even for trained analysts to distinguish from actual photographs, and some content was now realistic enough to be treated as real imagery under UK law.

The IWF warned that the technology was only improving and would pose more obstacles for watchdogs and law enforcement agencies to tackle the problem.

The research comes ahead of the UK hosting the AI safety summit next week, where world leaders and tech giants will discuss the developing issues around artificial intelligence.

In its latest research, the IWF said it had also found evidence of the commercialisation of AI-generated imagery, and warned that the technology was being used to “nudify” images of children whose clothed images had been uploaded online for legitimate reasons.

In addition, it said AI image tech was being used to create images of celebrities who had been “de-aged” and depicted as children in sexual abuse scenarios.

In a single month, the IWF said it investigated 11,108 AI images which had been shared on a dark web child abuse forum.

Of these, 2,978 were confirmed as images which breached UK law and 2,562 were so realistic it said they would need to be treated the same as if they were real abuse images.

Susie Hargreaves, chief executive of the IWF, said: “Our worst nightmares have come true. Earlier this year, we warned AI imagery could soon become indistinguishable from real pictures of children suffering sexual abuse, and that we could start to see this imagery proliferating in much greater numbers. We have now passed that point.

“Chillingly, we are seeing criminals deliberately training their AI on real victims’ images who have already suffered abuse.

“Children who have been raped in the past are now being incorporated into new scenarios because someone, somewhere, wants to see it.

“As if it is not enough for victims to know their abuse may be being shared in some dark corner of the internet, now they risk being confronted with new images, of themselves being abused in new and horrendous ways not previously imagined.

“This is not a hypothetical situation. We’re seeing this happening now. We’re seeing the numbers rise, and we have seen the sophistication and realism of this imagery reach new levels.

“International collaboration is vital. It is an urgent problem which needs action now. If we don’t get a grip on this threat, this material threatens to overwhelm the internet.”

The IWF said it feared that a deluge of AI-generated content could divert resources from detecting and removing real abuse, and in some instances could lead to missed opportunities to identify and safeguard real children.

By Press Association

More Technology News

See more More Technology News

App Stock

Spotify to axe 1,500 jobs to cut costs


David Hockney unveils Christmas display to bring ‘joy and hope’ to Londoners

Internet Browsing Stock

Review set to scrutinise AI’s impact on porn industry

Online Fraud Charter

Top tech firms sign new Government Charter to crack down on online fraud

Microsoft offices

Microsoft to invest £2.5bn in UK to boost AI plans

Musk accused companies like Apple and Disney of engaging in the ad boycott and trying to blackmail him.

Elon Musk launches expletive rant at advertisers who boycotted his social media platform X

Technology stock

Apple names its App Store apps of the year


Generative AI ‘helping criminals create more sophisticated cyber attacks’

Someone using a mobile phone

Virgin Media O2 expands national databank scheme for Christmas

Social media apps on a smartphone

Too much social media linked to harmful behaviours in teenagers, study suggests

Bank accounts

New data powers ‘could allow DWP to snoop on pensioners’ bank accounts’

AI image generators

Teach children about using AI at secondary school, IT professionals say

Child with a laptop

Social media sites failing to stop the spread of suicide content, charity says

Technology Stock

Public warnings about AI misinformation needed before election, peers told

Smartphone with Adobe logo on the screen

£16bn Adobe takeover could harm competition in digital design market – watchdog

Person using a laptop

No ‘smoking gun’ linking mental health harm and the internet – study