MindEval2
A clinical-grade evaluation framework for mental health AI, composed of two components: the MHQR evaluation scale and the MindSim patient simulation. It supports longitudinal assessment.
Timeline (last 90 days)
MindEval2 is a mental health AI evaluation framework composed of two parts: the Mental Health Quality Rubric (MHQR) for evaluation and MindSim for patient simulation.
MHQR was designed by the research team in collaboration with PhD-level psychologists, decomposing response quality into explicit, actionable clinical scoring items.
MindSim is an LLM-driven simulation framework that produces more diverse and more challenging patient interactions by combining carefully designed patient profiles with tools.
MindSim includes a memory component that is updated between each interaction, enabling MindEval2 to model the longitudinal dimension of mental health problems.
The research team evaluated 18 proprietary and open-weight models on MindEval2.
In the MindEval2 evaluation, frontier models have already clearly improved on surface quality and boundary issues, but guidance calibration is the most dominant failure mode across all models, accounting for most of the quality gap.
All Facts (6)
MindEval2 is a mental health AI evaluation framework composed of two parts: the Mental Health Quality Rubric (MHQR) for evaluation and MindSim for patient simulation.
50%UnverifiedMHQR was designed by the research team in collaboration with PhD-level psychologists, decomposing response quality into explicit, actionable clinical scoring items.
50%UnverifiedMindSim is an LLM-driven simulation framework that produces more diverse and more challenging patient interactions by combining carefully designed patient profiles with tools.
50%UnverifiedMindSim includes a memory component that is updated between each interaction, enabling MindEval2 to model the longitudinal dimension of mental health problems.
50%UnverifiedIn the MindEval2 evaluation, frontier models have already clearly improved on surface quality and boundary issues, but guidance calibration is the most dominant failure mode across all models, accounting for most of the quality gap.
50%UnverifiedThe research team evaluated 18 proprietary and open-weight models on MindEval2.
50%