AI deepfakes are creating sexual images of children at an alarming scale: Unicef

The UN children's agency on Wednesday highlighted a rapid rise in the use of artificial intelligence to create sexually explicit images of children, warning of real harm to young victims caused by the deepfakes.

According to a Unicef-led investigation in 11 countries, at least 1.2 million children said their images were manipulated into sexually explicit deepfakes – in some countries at a rate equivalent to "one child in a typical classroom" of 25 students.

The findings underscored the use of "nudification" tools, which digitally alter or remove clothing to create sexualized images.

"We must be clear. Sexualised images of children generated or manipulated using AI tools are child sexual abuse material," Unicef said in a statement.

"Deepfake abuse is abuse, and there is nothing fake about the harm it causes."

More from Tech

The agency criticized AI developers for creating tools without proper safeguards.

"The risks can be compounded when generative AI tools are embedded directly into social media platforms where manipulated images spread rapidly," Unicef said.

Elon Musk's AI chatbot Grok has been hit with bans and investigations in several countries for allowing users to create and share sexualized pictures of women and children using simple text prompts.

Unicef’s study found that children are increasingly aware of deepfakes.

"In some of the study countries, up to two-thirds of children said they worry that AI could be used to create fake sexual images or videos. Levels of concern vary widely between countries, underscoring the urgent need for stronger awareness, prevention, and protection measures," the agency said.

Unicef urged "robust guardrails" for AI chatbots, as well as moves by digital companies to prevent the circulation of deepfakes, not just the removal of offending images after they have already been shared.

Legislation is also needed across all countries to expand definitions of child sexual abuse material to include AI-generated imagery, it said.

The countries included in the study were Armenia, Brazil, Colombia, Dominican Republic, Mexico, Montenegro, Morocco, North Macedonia, Pakistan, Serbia, and Tunisia.

(Except the headline, this story has not been edited by Firstpost staff.)