AI deepfakes epidemic in schools with children as young as 10 suspended amid rising online abuse in classrooms

In 2023/24, Department for Education data shows a record 11,614 suspensions were handed to pupils using apps like Instagram, TikTok and Twitter to bully their peers or share inappropriate content.

This marks an increase of over 75% since 2021.

LBC has learned pupils are using so-called “nudify” apps to turn photos of their teachers and classmates into highly realistic deepfake pornographic images - a disturbing development that has prompted a leading internet safety charity to call for a total ban.

While such images may be fabricated, experts warn the effect on victims can be “devastating”.

Earlier this year, author Laura Bates, who has documented the extensive spread of online misogyny, warned parliament’s Women and Equalities Committee that the spread of deepfake porn is "the next sexual violence epidemic" facing classrooms.

Speaking to LBC, Ms Bates added that primary school children were being targeted by the technology: “We're seeing real world bullying happening as a result of these digital images that have been shared, there is a very real impact.

“People will say the images aren't real, so it doesn't matter, it doesn't count. But we're looking at girls of 10 and 11 years old who are dropping out of school, who are feeling unable to leave the house, young women who are developing PTSD as a result of this, it is a very, very real form of abuse and the impact is very real as well.”

Mum-of-two Sophie Parrish, 33, was sickened when she found out her brother-in-law created and disseminated deepfake pornographic images of her from when she was just a teenager.

“The vast majority were photos taken off my Instagram or Facebook from the age of 18 to 29," Sophie, a florist from Ainsdale, told LBC.

"The originals had been uploaded onto these sites, onto this forum, and then various men had deepfaked them, and people were commenting and describing what they’d do to me... it was hard to comprehend how and why it had happened”

Sophie worries about young people failing to grasp the impact such images can have on the victim: “I'll be honest, I was not expecting it to be 10- and- 11-year-olds. I would have thought more 15 or 16. But, from a younger age, we clearly need to be addressing that this is not okay.

"We would tell a child rape is wrong. We equally need to be as blunt as deepfaking somebody in this manner is very wrong. It's illegal... [but] there are still a lot tech companies that need to be held more accountable.

“My kids can work my phone better than I can, which is really scary when they're seven and nine."

Developers have faced criticism for the creation of nudify apps.

Indeed, in May, LBC exposed one app’s "horrifying and dangerous" decision to advertise how their product could place children’s faces on sexualised adult content.

With the technology rapidly evolving, teaching unions are calling for greater support to help them deal with incidents of deepfake porn emerging in their schools.

Margaret Mulholland, a spokesperson for the Association of School and College Leaders, warned that teachers have had to confront pupils over “spate of TikTok imagery”, as well as the spread of deepfakes.

“I know that this anecdotally is now being picked up, and that these deepfakes can include images of teachers,” Ms Mulholland told LBC. “We're really clear about how it's impacting on the well being of both staff and of young people.”

LBC’s analysis also revealed that 116 pupils were permanently excluded from school for their social media misuse in 23/24 - a fourfold increase on three years earlier.

Although suspensions were far more regular in secondary schools, 465 primary school pupils were also punished.

The Internet Watch Foundation (IWF), a leading online safety charity, supports victims of child sexual abuse imagery, many of whom have either sent nude images to their peers or have had deepfakes of them created.

“We've seen a massive rise in AI-generated material and even just the development in the technology over the last two years,” says Hannah Swirsky, the IWF’s Head of Policy.

“Over the last two years, we've seen offenders publish manuals online on how to create this material - this is freely available on the open web.”

Hannah’s warning is backed up by striking data. In the first six months of this year, the charity witnessed a five-fold increase in child abuse imagery created by AI.

The IWF is calling for an outright ban on nudify apps.

Together with the National Crime Agency (NCA), the charity has published guidance for teachers on how to handle reports of the creation and spread of AI-generated child sexual abuse imagery.

Alastair Simpson, child abuse lead for the National Police Chiefs’ Council, added that the creation of this imagery can leave children vulnerable to financial exploitation.

In response to LBC’s findings, a Department for Education spokesperson said: “The new RSHE guidance responds to the realities facing children today, making sure that all young people understand healthy relationships, sexual ethics beyond consent and harmful online content such as pornography and deepfakes.

“We know that there are issues with children’s online experiences, which is why we have guidance on banning mobile phones during the school day to limit the disruption in the classroom and we have brought in better protections from harmful content through the Online Safety Act.”

Victims of child sexual abuse can confidentially report images to the Internet Watch Foundation using their Report Remove Tool