Unverified50% confidenceOpinionExact time
Existing mental health AI benchmarks typically have a large model simulate a patient in multi-turn dialogue and then use another LLM as a judge to score, but the simulated patients are often overly cooperative and overly self-analytical, and the evaluation criteria often rely on opaque Likert scales.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/16/2026
First Seen
Valid until: 1/14/2027
Sources
Related Entities
Related Claims
UnverifiedThe limitations of LLM-as-a-judge in mental health scenarios include: the judge model may lack clinical knowledge, the basis for scoring is not traceable, it is easily affected by superficial features such as wording and length, and there is a lack of a clear mapping between scores and reasons.69% similarUnverifiedHallucination in the psychiatric interview setting manifests as inferring that a patient has a certain symptom, risk, or diagnostic tendency without a basis in the dialogue.66% similarUnverifiedIf training data underrepresents certain populations, AI diagnostic recommendations may produce systematic bias against specific ethnic groups, genders, or age groups.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1041923API
curl https://kongchang.com/api/v1/knowledge/claims/1041923MCP
get_claim(id=1041923)