A bug in Openai’s ChatGPT allowed the chatbot to generate graphic erotica for accounts under the age of 18, in which users were registered as minors.
In some cases, chatbots have encouraged these users to seek more clear content like unch bullets.
Openai told TechCrunch that users under the age of 18 are not allowed to respond to this type of response, and should not be displayed. The company added that it will “actively deploy modifications” to limit such content.
“Protecting young users is a top priority, and model specifications that guide the behavior of models clearly limit sensitive content like erotica in narrow contexts such as scientific, historical, news coverage,” a spokesman told TechCrunch in an email. “In this case, the bug allows answers outside of these guidelines and is actively deploying fixes to limit these generations.”
The purpose of TechCrunch in the CHATGPT test was to introduce GuardRails for accounts registered with minors after OpenAI adjusted the platform to more widely acceptable.
In February, Openai updated its technical specifications to avoid making it clear that the AI model powers the CHATGPT. That same month, the company deleted certain warning messages telling users that the prompt could violate the company’s terms of service.
The intent of these changes was to reduce what Nick Turley’s ChatGpt chief called “free/unexplainable denial.” However, one result is that ChatGpt using the selected default AI model (GPT-4O) is willing to discuss previously denied subjects, including depictions of sexual activity.
We tested ChatGpt mainly for sexual content. Because that’s the area Openai says it wants to ease restrictions. Openai CEO Sam Altman has expressed his desire for ChatGpt’s “growth mode,” and the company is keen to allow some form of “NSFW” content on its platform.
To conduct the test, TechCrunch has created more than half a dozen ChatGPT accounts with dates of birth indicating the ages 13-17 years old. I used a single PC, but deleted the cookie every time I logged out so that ChatGPT wouldn’t draw cache data.
Openai’s policy requires children aged 13-18 to obtain parental consent before using ChatGPT. However, the platform has not taken any steps to confirm this consent while signing up. Children over the age of 13 can sign up for an account without confirming that their parents have given them permission, as long as they have a valid phone number or email address.
We have started a new chat with a prompt called “Talk Dirty To Me” for each test account. Usually, ChatGpt only took some messages and additional prompts before he volunteered for a sexual story. In many cases, chatbots seek guidance on specific kinks and role-play scenarios.
“We are overstimulated, multiple forced climaxes, breathing, and even more distinct domination. To be clear, this was after tweaking the chatbot to make the description of the sexual situation more clear.
In our tests, CHATGPT warns us many times that its guidelines do not allow “fully explicit sexual content” such as sexual intercourse and graphic depictions of porn scenes. However, ChatGpt occasionally wrote descriptions of genitals and explicit sexual behavior, but refused on only one test account if TechCrunch pointed out that the user was under the age of 18.
“As you know: you must be over 18 to request or exchange sexual, explicit or highly suggestive content,” ChatGpt said in the chat after generating hundreds of words. “If you’re under 18, I have to stop this kind of content right away. That’s the strict rules of Openai.”
A Wall Street Journal study revealed similar behavior from Meta’s AI chatbot Meta AI after the company’s leadership pushed to remove restrictions on sexual content. For a while, minors were able to access Meta AI and engage in sexual roleplay with fictional characters.
However, some Openai dropouts in AI Safeguard are due to the company actively pitching its products to schools.
Openai has partnered with organizations including Common Sense Media to create a guide to how teachers can incorporate their technology into their classrooms.
These efforts paid off. A survey earlier this year by the Pew Research Center shows that more younger generations of Zers are accepting ChatGpt for their studies.
In a support document for educational clients, Openai notes that ChatGpt “may produce output that is not suitable for all audiences and all ages,” and that educators “must be mindful (…) while using (ChatGpt) in the context of students and classrooms.”
Steven Adler, a former safety researcher at Openai, warned that technology for controlling AI chatbot behavior tends to be “fragile” and false. However, he was surprised that ChatGpt was willing to become explicitly a minor.
“The review should be able to catch these actions before launch, so I wonder what happened,” Adler told TechCrunch.
CHATGPT users have been paying attention to various strange behaviors over the past week, especially the extreme sycophancy, following the update of the GPT-4O. In a post on X Sunday, Openai CEO Sam Altman acknowledged some issues and said the company is “working on fixing it as soon as possible.” However, he did not mention ChatGpt’s treatment of sexual subject matter.