Unverified50% confidenceOpinionExact time
The limitations of LLM-as-a-judge in mental health scenarios include: the judge model may lack clinical knowledge, the basis for scoring is not traceable, it is easily affected by superficial features such as wording and length, and there is a lack of a clear mapping between scores and reasons.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedHallucination in the psychiatric interview setting manifests as inferring that a patient has a certain symptom, risk, or diagnostic tendency without a basis in the dialogue.71% similarUnverifiedExisting mental health AI benchmarks typically have a large model simulate a patient in multi-turn dialogue and then use another LLM as a judge to score, but the simulated patients are often overly cooperative and overly self-analytical, and the evaluation criteria often rely on opaque Likert scales.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1041929API
curl https://kongchang.com/api/v1/knowledge/claims/1041929MCP
get_claim(id=1041929)