New Law Gives Tech Companies and Child Safety Agencies a Tool to Combat AI-Generated Abuse Images
The UK government has introduced new legislation that allows tech companies and child safety agencies to test artificial intelligence tools for their potential to create child abuse images. This move comes as the number of reports of AI-generated child sexual abuse material (CSAM) has more than doubled in the past year, with 426 cases reported in 2025 compared to 199 in 2024.
The law change will give designated companies and agencies permission to examine AI models, such as those used by chatbots like ChatGPT and image generators like Google's Veo 3, under strict conditions. The aim is to prevent the creation of CSAM at source and help authorities spot potential risks early on.
Experts welcome this move, saying it will ultimately stop abuse before it happens. However, critics argue that AI-generated CSAM has become a significant problem, with girls making up 94% of illegal images in 2025 and depictions of newborns to two-year-olds rising significantly.
The Internet Watch Foundation reports that instances of category A material have more than doubled, with new cases involving "potentially limitless amounts of sophisticated, photorealistic child sexual abuse material". This has made it difficult for survivors, who are often victimized again when they see AI-generated content.
Childline, a UK helpline for children, has also seen an increase in counselling sessions related to AI and online bullying. The charity warns that AI-faked images and chatbots can be used to blackmail children and dissuade them from reporting abuse.
While the law is aimed at preventing the creation of CSAM at source, critics argue that it may not go far enough to address the root causes of online child abuse. Nevertheless, this move represents a significant step towards making AI products safer before they are released into the public domain.
The UK government has introduced new legislation that allows tech companies and child safety agencies to test artificial intelligence tools for their potential to create child abuse images. This move comes as the number of reports of AI-generated child sexual abuse material (CSAM) has more than doubled in the past year, with 426 cases reported in 2025 compared to 199 in 2024.
The law change will give designated companies and agencies permission to examine AI models, such as those used by chatbots like ChatGPT and image generators like Google's Veo 3, under strict conditions. The aim is to prevent the creation of CSAM at source and help authorities spot potential risks early on.
Experts welcome this move, saying it will ultimately stop abuse before it happens. However, critics argue that AI-generated CSAM has become a significant problem, with girls making up 94% of illegal images in 2025 and depictions of newborns to two-year-olds rising significantly.
The Internet Watch Foundation reports that instances of category A material have more than doubled, with new cases involving "potentially limitless amounts of sophisticated, photorealistic child sexual abuse material". This has made it difficult for survivors, who are often victimized again when they see AI-generated content.
Childline, a UK helpline for children, has also seen an increase in counselling sessions related to AI and online bullying. The charity warns that AI-faked images and chatbots can be used to blackmail children and dissuade them from reporting abuse.
While the law is aimed at preventing the creation of CSAM at source, critics argue that it may not go far enough to address the root causes of online child abuse. Nevertheless, this move represents a significant step towards making AI products safer before they are released into the public domain.
I'm literally shook by how fast AI-generated CSAM is on the rise 
Like what even is going on here?! 
426 cases in just one year?! That's insane! 
. This law change better be doing something serious about addressing the root causes of online child abuse or else I'm gonna lose it 
.
. And what about all the AI models that aren't even mentioned here? Are we just gonna sit back and let them go unchecked?! 
. It's like these companies and tech firms need to do more than just test AI tools for potential abuse - they should be working on how to prevent this stuff from happening in the first place 
.
. I just wish there was more being done to address this issue and make sure AI products are safer before they're released into the public domain 
.
it's like 90% of the time its those AI generated pics thats the one that gets the kids in trouble, and now we got tech companies and child safety agencies workin together to test these AI models its a step in the right direction, but we gotta keep pushin for more, cause if we dont, we'll just see this problem get even bigger 
what can't be stopped online is still gonna be out there waiting for us
. Need stricter laws ASAP 
, not just tech companies testing AI tools 
. Can't let them release this trash into the wild 
But at the same time, 426 cases reported in 2025? That's still way too many... and what really breaks my heart is that girls make up 94% of those images. We need to keep pushing for more solutions, like education programs and support services for our little ones. And let's not forget about the impact on survivors who are often re-victimized when they see AI-generated content... it's just heartbreaking 
The UK government thinks testing AI models is a good start, but have they thought about the bigger picture? We're talking AI-powered algorithms that can create images, not just random glitches.
. What's the point of having a law if it's just gonna be watered down by bureaucrats and lawyers?
.
meanwhile we're still gonna have these AI-generated abusers getting away scot-free, and the ones who are caught will just get a slap on the wrist 
the real victims are still gonna suffer, kids seeing all this stuff online and being traumatized over and over again 
. The fact that girls are making up 94% of these images is just heartbreaking 
. We can't just focus on tech companies and AI models; we need to address the societal issues that lead to online abuse in the first place 
.