European Commission proposes criminalizing AI-powered child abuse

The widespread growth of generative AI has provided lawmakers with a clear motivation to review the regulations to facilitate crime reporting for child sexual abuse victims and provide relevant support.

The European Commission has announced that it is considering criminalizing artificial intelligence (AI)-generated imagery and deepfakes portraying child sexual abuse (CSA) to update laws in response to technological advancements.

The proposal includes introducing a new criminal offense for live-streaming child sexual abuse. It also seeks to criminalize the possession and exchange of “pedophile manuals.” The proposal also mentions AI chatbots used for child abuse.

The commission aims to update 2011 rules with the proposal, suggesting changes to mandatory reporting of offenses. These measures are part of a broader package aimed at preventing child sexual abuse, enhancing online risk awareness, facilitating crime reporting for victims and providing support, including the right to financial compensation.

Child sexual abuse is a heinous crime which has evolved significantly over the past years.

Today, we are adopting a proposal to update the criminal law rules on child sexual abuse and sexual exploitation.

Learn more → https://t.co/mtz68GuEJJ#SecurityEU pic.twitter.com/QF7TsFX7cy

— European Commission (@EU_Commission) February 6, 2024

The commission’s impact assessment conducted before the proposal highlighted increased online presence and “latest technological developments” as factors creating new opportunities for child sexual abuse. To address online safety risks for children, the proposal encourages member states to invest in “awareness raising.”

There are worries that the technology is misused to create child sexual abuse material (CSAM), with concerns rising that this artificial content could complicate the identification of real victims for law enforcement authorities and highlighting the need to review existing regulations.

Related: EU member states ‘endorse political agreement’ to advance AI Act

However, the final form of the proposals will be decided by the European Parliament and the European Council. According to the commission, upon reaching an agreement on amending the current directive on combating CSA, it would come into effect 20 days after publication in the Official Journal of the EU.

According to the ordinary legislative procedure, when the European Commission submits a proposal, the European Parliament and the Council of Ministers may approve or amend it. If disagreements persist, both can propose amendments. Negotiations follow, and upon conclusion, both institutions can vote for or against the final version.

In May 2022, the commission proposed additional legislation focusing on digital services’ obligation to employ automated technologies for detecting and reporting CSAM and identifying grooming activity targeting children. According to the commission, the current proposal complements the 2022 regulation proposal.

However, there has been a significant increase in awareness of the dangers of deepfakes and AI-generated images in less than two years since the private message scanning proposal was introduced.