SAFE-Teen
A systematic study evaluating the safety of chatbots in adolescent mental health dialogue scenarios. It employs clinician-involved evaluation methods, simulating conversations with adolescents aged 13 to 17 and conducting multi-turn interaction tests against 22 safety criteria.
Timeline (last 90 days)
The SAFE-Teen study covered 18 chatbot APIs and evaluated a total of 32,555 simulated conversations.
The simulated conversations were based on 106 seed scenarios, from which 983 variants were derived.
The evaluation used 22 safety criteria.
Without teen-aware prompts, the overall safety criterion failure rate was 43.3%.
When adolescents exhibit pushback behavior, safety failure rates on mental health diagnosis, medication use and substance abuse boundaries rise by 10-12 percentage points (under the no-prompt condition).
Even with adolescent-aware prompts, safety failure rates under pushback scenarios remain high.
The SAFE-Teen study is one of the largest safety evaluations of adolescent mental health chatbots to date.
All Facts (7)
The SAFE-Teen study covered 18 chatbot APIs and evaluated a total of 32,555 simulated conversations.
50%UnverifiedThe simulated conversations were based on 106 seed scenarios, from which 983 variants were derived.
50%UnverifiedThe evaluation used 22 safety criteria.
50%UnverifiedWithout teen-aware prompts, the overall safety criterion failure rate was 43.3%.
50%UnverifiedWhen adolescents exhibit pushback behavior, safety failure rates on mental health diagnosis, medication use and substance abuse boundaries rise by 10-12 percentage points (under the no-prompt condition).
50%UnverifiedEven with adolescent-aware prompts, safety failure rates under pushback scenarios remain high.
50%UnverifiedThe SAFE-Teen study is one of the largest safety evaluations of adolescent mental health chatbots to date.
50%