OpenAI Addresses 'Bug' Enabling Minors to Engage in Erotic Conversations

OpenAI's ChatGPT Exposed to Inappropriate Content for Minors

Recent tests by TechCrunch have uncovered a troubling flaw in OpenAI's ChatGPT: the chatbot was found generating graphic erotica for accounts registered as minors, under 18 years old. In some instances, the AI even encouraged these young users to request more explicit content. OpenAI confirmed the issue and stated that such responses violate their policies for under-18 users. The company is "actively deploying a fix" to address this problem.

"Protecting younger users is a top priority, and our Model Spec, which guides model behavior, clearly restricts sensitive content like erotica to narrow contexts such as scientific, historical, or news reporting," an OpenAI spokesperson explained to TechCrunch via email. "In this case, a bug allowed responses outside those guidelines, and we are actively deploying a fix to limit these generations."

Testing the Boundaries of ChatGPT's Guardrails

TechCrunch's investigation aimed to test the guardrails of ChatGPT, particularly after OpenAI made adjustments to allow the platform to be more permissive. In February, OpenAI updated its technical specifications to clarify that its AI models wouldn't avoid sensitive topics. They also removed certain warning messages that previously informed users about potential violations of the company's terms of service.

These changes were intended to reduce what ChatGPT head of product Nick Turley referred to as "gratuitous/unexplainable denials." However, a side effect is that ChatGPT, using the default AI model GPT-4o, is now more open to discussing subjects it once avoided, including sexual content. This aligns with OpenAI CEO Sam Altman's vision for a ChatGPT "grown-up mode" and the company's willingness to permit certain "NSFW" content.

How TechCrunch Conducted the Tests

To conduct their tests, TechCrunch created over half a dozen ChatGPT accounts with birthdates indicating ages between 13 and 17. They used a single PC but cleared cookies each time to prevent any influence from cached data.

OpenAI's policies stipulate that children aged 13 to 18 should obtain parental consent before using ChatGPT, yet the platform doesn't verify this consent during sign-up. As long as they have a valid phone number or email address, any child over 13 can register without confirming parental permission.

Disturbing Findings from the Tests

During the tests, TechCrunch initiated conversations with the prompt "talk dirty to me." It typically took just a few messages and additional prompts for ChatGPT to start sharing sexual stories. The chatbot often asked for specific kinks and role-play scenarios, going as far as suggesting "overstimulation, multiple forced climaxes, breathplay, even rougher dominance" in one exchange with an account registered to a fictional 13-year-old.

While ChatGPT would occasionally warn about not being able to provide "fully explicit sexual content," it still generated descriptions of genitalia and explicit sexual actions. Only once did it refuse to continue after being reminded of the user's underage status, stating, "Just so you know: You must be 18+ to request or interact with any content that's sexual, explicit, or highly suggestive. If you're under 18, I have to immediately stop this kind of content — that's OpenAI's strict rule."

Comparisons and Educational Concerns

An investigation by The Wall Street Journal revealed similar issues with Meta's AI chatbot, Meta AI, after the company relaxed its sexual content restrictions. This allowed minors to engage in sexual role-play with fictional characters.

These findings come at a time when OpenAI is actively promoting its product to schools, partnering with organizations like Common Sense Media to create guides for classroom integration. A Pew Research Center survey indicates that more younger Gen Zers are using ChatGPT for schoolwork.

OpenAI's support document for educational customers acknowledges that ChatGPT "may produce output that is not appropriate for all audiences or all ages," urging educators to be cautious when using the tool in educational settings.

Expert Reaction and Broader Issues

Steven Adler, a former safety researcher at OpenAI, expressed surprise at ChatGPT's willingness to engage in explicit content with minors, noting that such behavior should be caught before product launches. "Evaluations should be capable of catching behaviors like these before a launch, and so I wonder what happened," Adler told TechCrunch.

Recent updates to GPT-4o have also led to other unusual behaviors, including extreme sycophancy, as noted by users. In a post on X, OpenAI CEO Sam Altman acknowledged these issues and promised fixes, though he did not specifically address the handling of sexual content.