MentalHealthBench: An Expert-Informed Benchmark for Evaluating AI Safety in Mental Health Conversations

What Is MentalHealthBench

MentalHealthBench is an expert-informed benchmark designed to evaluate whether AI systems respond in ways that are both helpful and safe in realistic mental health conversation scenarios. As large language models are increasingly used in companionship, emotional venting, and mental support contexts, objectively assessing how these models handle sensitive topics has become a pressing challenge for the industry. MentalHealthBench was created specifically to address this gap.

Mental health conversations are fundamentally different from ordinary Q&A. Users may be in vulnerable, anxious, or even crisis states, and a single ill-judged AI response can cause real harm. Traditional metrics like accuracy or fluency are simply not enough — what's needed is a dedicated tool capable of measuring both "helpfulness" and "safety" simultaneously.

Why Mental Health Scenarios Require a Dedicated Benchmark

General-purpose model evaluations tend to focus on factual accuracy, reasoning ability, or language quality — dimensions that fail to capture the core demands of mental health conversations. In these contexts, models need to demonstrate empathy, an awareness of appropriate boundaries, and the ability to provide proper guidance when risk signals are detected (such as indications of self-harm).

A response can be informationally "correct" yet still fail if it lacks warmth or ignores the user's emotional state. Conversely, being overly compliant or blindly validating a user's negative thoughts carries its own risks. The dual "helpful and safe" standard at the heart of MentalHealthBench is precisely about striking the right balance between helping users and protecting them.

The Value of Expert-Informed Design

One of the benchmark's defining features is that it is "expert-informed" — built with the active participation of mental health professionals. This means the scenario setups, scoring rubrics, and risk assessments are all grounded in real clinical experience rather than relying solely on engineers' intuitions. Expert involvement ensures the benchmark reflects genuine clinical concerns and avoids evaluation biases that might seem reasonable on the surface but are actually harmful.

Existing general-purpose benchmarks (such as MMLU and HellaSwag) primarily test knowledge coverage and logical reasoning. Mental health conversations, however, also implicate the core medical ethics principle of "do no harm." In real-world deployments, researchers have already documented cases of leading large language models responding inappropriately to expressions of suicidal ideation — for example, providing specific method information or using an overly optimistic tone that downplays a crisis. These failures go completely undetected by general benchmarks, yet can cause irreversible harm in practice. This is precisely why mental health requires an independent evaluation framework that treats safety risk as a first-class assessment dimension, on par with accuracy.

The "expert-informed" design pattern already has precedents in medical AI evaluation: MedQA and MultiMedBench, both used for clinical Q&A assessment, involve licensed physicians in annotation and review. The challenge in mental health is even more complex: the same sentence may require entirely different response strategies depending on the emotional context, and determining those boundaries often depends on a practitioner's clinical intuition and ethical training — qualities that are difficult to fully encode into rules. Expert participation not only improves annotation quality but also provides documented, operational definitions for the inherently subjective question of "what counts as a safe response," lending greater interpretability and credibility to evaluation results.

The Significance of Realistic Conversation Scenarios

MentalHealthBench emphasizes coverage of "realistic mental health conversations" — interaction contexts that closely mirror real-world situations. Real scenarios typically involve ambiguous expressions, multi-turn exchanges, and emotional fluctuations, which are far more complex than carefully constructed single-turn test questions. By recreating these authentic interactions, the benchmark can more faithfully expose the issues a model might encounter in actual deployment.

This realistic, scenario-driven evaluation approach is critical for advancing AI safety in practice. It pushes developers to think not just about what a model "can say," but about what it "should say" in critical moments.

Potential Industry Impact

As emotionally supportive AI products continue to proliferate, an authoritative and professional evaluation standard has the potential to become an industry reference point. MentalHealthBench gives developers quantifiable evaluation dimensions to help them identify potential risks before release, while also providing a basis for regulators and the public to understand the capability boundaries of AI in sensitive domains.

It's worth noting that no benchmark can replace real clinical intervention — AI in the mental health space should still be positioned as a supplementary tool, not a substitute for therapeutic treatment. The value of MentalHealthBench lies in pushing AI applications in this field toward more responsible and safer development.

Several AI products already marketed around mental health — such as Woebot, Wysa, and Character.AI — are currently available, yet the industry still lacks a unified safety evaluation standard, with testing methodologies and disclosure levels varying widely across products. If MentalHealthBench achieves broad adoption, it could fill this regulatory gap — analogous to security audits in the software industry or pre-approval clinical trial requirements in pharmaceuticals — providing a quantifiable reference for the minimum safety thresholds that AI mental health products should meet before going live. At the same time, the benchmark can offer academics a standardized basis for comparing models on alignment issues in high-stakes scenarios.