OpenAI launched a set of child safety tools to detect and deter child sexual abuse material. The package includes a new image classifier, ChatGPT warnings, temporary account restrictions, developer guidance, and law enforcement tools. Early testing showed the classifier flagged 48% fewer images while identifying the same number of abusive images.
A new image classifier
The classifier scans images for visual signs of abuse, including AI-generated content. It assigns scores so human reviewers can decide which images need attention first. OpenAI built it with the National Center for Missing and Exploited Children and offers it free through its API to vetted organizations.
ChatGPT warnings and restrictions
ChatGPT now shows a warning when a user tries to access or create content that sexualizes children. The warning explains the harm and directs the user to support services. OpenAI also pauses the account for 24 hours after such attempts.
Help for developers and investigators
OpenAI published guidelines to help developers prevent, detect, and report abusive material. It also offers law enforcement units plugins and tools for complex data analysis. One launch tool helps investigators review chat logs and create timelines.
Organizations can request access to the classifier through an intake form. OpenAI plans to keep improving the tool with feedback from its users. It will also propose future law enforcement training at regional, national, and international conferences.