AGENTTechnology and AI
← Latest editionCommon Sense rates ChatGPT for Teens an unacceptable risk
Common Sense Media says tests of more than 4,000 prompts found gaps in crisis referrals and parent alerts. OpenAI disputes whether the tests reflect how its safeguards work in practice.
- The Youth AI Safety Institute tested ChatGPT before and after its teen experience launched on 18 August.
- In its post-launch crisis tests, the chatbot provided some form of resource for 74% of prompts its advisers had judged warranted one.
- In a separate test, newly created teen accounts linked to parents produced no alerts during explicit conversations about suicide, self-harm or disordered eating.
- OpenAI challenged the assessment’s methods and said its larger-scale data showed more hotline resources displayed to under-18 users.
Common Sense Media’s Youth AI Safety Institute has rated ChatGPT for Teens an “Unacceptable Risk” and urged OpenAI to stop marketing ChatGPT to teenagers until it can demonstrate the safeguards work. The rating is the Institute’s judgement, based on scripted tests of the chatbot—not a measure of real-world injury rates or proof that a particular teenager was harmed.1
The Institute ran more than 4,000 prompts across two testing periods: before the teen experience was announced and after it launched on 18 August. In its mental-health tests, three child and adolescent psychiatrists had first identified which of 390 prompts warranted a crisis resource. For the 201 prompts that met that threshold, the Institute says post-launch responses provided a hotline, named professional or general medical resource in 74% of cases. A hotline appeared in 23%; a specific medical or mental-health professional was recommended in 58%.1
The Institute’s own minimum threshold for detecting clear crisis disclosures is 95%, informed by clinical guidance and expert input. It says ChatGPT fell below that threshold in three of five severe-harm categories it tested: suicide and self-harm, impaired reality, and disordered eating. That 95% is the Institute’s benchmark, not a regulatory standard or an estimate of how often teens will encounter these situations.12
The parent-alert findings came from two different tests. In a dedicated experiment, researchers created more than a dozen new free accounts registered to teenagers, linked each to a parent account and began with no chat history. The Institute says those accounts produced no alerts during conversations containing explicit disclosures of suicide, self-harm or disordered eating, including tests lasting up to an hour. In its broader post-launch testing, two alerts arrived on an account that had already been used for hundreds of prompts over several weeks.1
OpenAI told Axios that linked accounts may need to be connected for about three hours before alerts work. The Institute says some of its test accounts were linked for less than three hours and others for longer, and stands by its findings. OpenAI also argued that age prediction draws on multiple signals and may take up to two weeks; it said company-wide data showed an increase in hotline-resource displays to under-18 users during the period. Those figures were not independently tested in the Institute’s assessment.13
The test was not a controlled experiment capable of isolating the effect of teen-mode settings. The pre- and post-launch periods came one after another, so changes to the underlying model could account for some differences. The Institute also did not calculate inter-rater agreement and did not test voice mode, image generation, personalisation or group chats.1
The assessment records successes as well as failures: the chatbot generally refused explicit sexual or romantic role-play, and its clinicians judged much of its crisis advice sound. The Institute’s concern is that safeguards around that advice—especially referrals and alerts—were inconsistent. OpenAI launched the teen experience with learning features and additional protections, including parental controls and safety notifications for some high-risk situations.14