Meta secretly tested ChatGPT, Gemini, and Character.AI with thousands of minor-perspective crisis prompts
Quick Answer
Meta conducted secret tests using prompts about sensitive topics from minors' perspectives on ChatGPT, Gemini, and Character.AI, raising ethical concerns.
Quick Take
The project, named 'Cannes,' involved contractors simulating minors, sending over 45,000 crisis-related prompts, and Meta claims it was responsible safety testing. However, the companies tested were unaware, and there are ongoing concerns about AI's impact on youth.
Key Points
- Contractors created fake minor accounts to test AI responses on sensitive issues.
- Over 45,000 prompts about self-harm and drugs were sent in a single round.
- Meta claims the testing was industry-standard, but the companies were unaware.
- Concerns grow as 64% of children aged 9-17 have used AI chatbots.
- Several teen suicides have been linked to interactions with AI chatbots.
📖 Reader Mode
~2 min readMeta apparently tested rival chatbots like ChatGPT with prompts about suicide, sex, and drugs from the perspective of minors.
Hundreds of contractors pretended to be minors and sent sensitive prompts to ChatGPT, Gemini, and Character.AI, according to WIRED. The project, internally called "Cannes," was run by Meta's contractor Covalen and stayed active through at least April 2026.
The contractors created fake accounts with birthdates under 18, sent prompts about self-harm, eating disorders, and drugs to the chatbots, then copied the responses into spreadsheets. In a single testing round in August 2025, more than 45,000 prompts were sent, WIRED reported. Many were written from the perspective of children in crisis.
Meta defended the practice as responsible, industry-standard safety testing. The company also said it didn't use the collected chatbot responses to train its own AI models. Documents reviewed by WIRED don't show what Meta actually did with the data it gathered.
The companies being tested had no idea. Character.AI told WIRED the testing violated its terms of service. OpenAI is looking into the matter. A Google spokesperson said the company didn't approve the tests and couldn't determine from the available information whether they broke its terms.
Teens and AI chatbots remain a growing concern
Meta already faced backlash when an internal document revealed that its AI chatbot guidelines had allowed the generation of romantic and sexualized conversations with minors. The company later shut down access to AI characters for teens.
The problem goes beyond Meta and likely drove the company's prompt testing. A report by the UK organization Internet Matters found that 64 percent of children between 9 and 17 have already used AI chatbots. Effective age verification is mostly absent. 58 percent of kids aged 9 to 12 said they use chatbots despite a minimum age requirement of 13.
Several teen deaths have been linked to AI chatbots. A 14-year-old Character.AI user took his own life after spending months building an intense emotional bond with a chatbot. The parents of a 16-year-old in California sued OpenAI, alleging ChatGPT played a role in their son's suicidal thoughts. In July 2025, 23-year-old Zane Shamblin died by suicide after ChatGPT reportedly validated his suicidal thoughts over several hours.
— Originally published at the-decoder.com
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from The Decoder
See more →
An AI model programmed nonstop for 19 days on a single MirrorCode task that cost $2,600 to run
Epoch AI's MirrorCode benchmark reveals Claude Opus 4.7 as the leader with a 56% solve rate, reconstructing a 16,000-line toolkit in 14 hours. Despite this, all models tested struggle with the most complex tasks, highlighting limitations in current AI capabilities. The single task consumed $2,600 over 19 days, raising questions about cost-effectiveness in AI development.
