Meta secretly tested ChatGPT, Gemini, and Character.AI with thousands of minor-perspective crisis prompts
Meta apparently tested rival chatbots like ChatGPT with prompts about suicide, sex, and drugs from the perspective of minors.
Hundreds of contractors pretended to be minors and sent sensitive prompts to ChatGPT, Gemini, and Character.AI, according to WIRED. The project, internally called "Cannes," was run by Meta's contractor Covalen and stayed active through at least April 2026.
The contractors created fake accounts with birthdates under 18, sent prompts about self-harm, eating disorders, and drugs to the chatbots, then copied the responses into spreadsheets. In a single testing round in August 2025, more than 45,000 prompts were sent, WIRED reported. Many were written from the perspective of children in crisis.
Meta defended the practice as responsible, industry-standard safety testing. The company also said it didn't use the collected chatbot responses to train its own AI models. Documents reviewed by WIRED don't show what Meta actually did with the data it gathered.
The companies being tested had no idea. Character.AI told WIRED the testing violated its terms of service. OpenAI is looking into the matter. A Google spokesperson said the company didn't approve the tests and couldn't determine from the available information whether they broke its terms.
Meta already faced backlash when an internal document revealed that its AI chatbot guidelines had allowed the generation of romantic and sexualized conversations with minors. The company later shut down access to AI characters for teens.
The problem goes beyond Meta and likely drove the company's prompt testing. A report by the UK organization Internet Matters found that 64 percent of children between 9 and 17 have already used AI chatbots. Effective age verification is mostly absent. 58 percent of kids aged 9 to 12 said they use chatbots despite a minimum age requirement of 13.
Several teen deaths have been linked to AI chatbots. A 14-year-old Character.AI user took his own life after spending months building an intense emotional bond with a chatbot. The parents of a 16-year-old in California sued OpenAI, alleging ChatGPT played a role in their son's suicidal thoughts. In July 2025, 23-year-old Zane Shamblin died by suicide after ChatGPT reportedly validated his suicidal thoughts over several hours.