LLM's "Empathy Bias": When Identity Disclosure Is Misread as the Root of Suffering

Audit finds LLMs rewrite confiders' identity disclosures into causes of suffering, forming insidious compassionate othering.
The study used paired counterfactual interventions to audit how six LLMs handle identity disclosures in suicidal ideation and distress scenarios. Models tend to turn identity such as gender, race, and sexual orientation into causal narratives explaining suffering, attributing distress to imagined identity-related adversity—often wrapped in warm empathy, termed "compassionate othering." This over-interpretation is more frequent for marginalized identities and may overshadow individuals' real circumstances. The study suggests grounding explanations of suffering in the person's own disclosure.
Title
LLM's "Empathy Bias": When Identity Disclosure Is Misread as the Root of Suffering
Meta Description
A new study audits how six LLMs interpret identity disclosures amid distress, finding that models often automatically convert identity into a cause of suffering, forming "compassionate othering"—marginalized identities are more likely to be over-interpreted.
Article Content
When a person confides psychological distress or even suicidal ideation to a large language model (LLM), they often inadvertently reveal their social identity—gender, race, sexual orientation, religious beliefs, disability status. This identity information should help the model provide more contextually sensitive support, but a new audit study finds that models are likely to "rewrite" these identity disclosures into a narrative that explains the suffering: your pain comes from your identity.
The study, titled "CompAIssionate Other: Identity Assumptions in LLM Responses to Distress," systematically evaluated six LLMs across two suicidal ideation and psychological distress datasets, 16 identity and control conditions, and two prompt protocols using paired counterfactual interventions. The conclusions are not optimistic: models transform identity disclosures into explanatory narratives, attributing distress to imagined identity-related adversity, and this assumption is often wrapped in seemingly warm empathic responses—especially when marginalized identities are disclosed.

Research Method: Using Counterfactual Pairing to "Interrogate" the Model's Attribution Logic
To determine whether a model truly treats identity as a cause of suffering, a single response is insufficient—empathic language itself can obscure attribution. The study adopted a paired counterfactual intervention strategy: for the same distress narrative, only the disclosed identity label is changed while all other content remains identical, and then the differences in model responses are compared. If changing the identity changes the model's explanation of the source of distress, then that attribution is identity-driven rather than based on what the person actually disclosed.
The audit covered six LLMs, two datasets, 16 identity and control conditions, and tested two prompt protocols to verify the robustness of the conclusions. The key to this design is "control": only when identity is the sole variable can the model's tendency to attribute distress to identity be clearly exposed.
Counterfactual intervention originates from causal inference methodology: while keeping all other conditions unchanged, only one variable is altered to observe whether the outcome changes, thereby determining whether that variable is a cause of the outcome. In LLM auditing, this typically manifests as constructing a pair of inputs that differ only in a certain attribute (such as an identity label), then comparing the differences in model outputs. Its advantage lies in pushing "correlation" toward "causality"—if merely changing the identity label can systematically change the model's explanation of the source of distress, then such attribution is difficult to explain away as "content differences" or "random fluctuation."
Prompt protocol refers to the prompt structure and interaction style set by researchers to elicit model outputs, such as whether a system role is given, whether step-by-step reasoning is required, whether response length is limited, etc. Different protocols may activate different behavioral patterns in the model, so repeating experiments under multiple protocols is a common practice for testing the robustness of conclusions.
Core Finding: Identity Is Automatically Translated into an "Adversity Script"
The most important finding of the study is that models tend to transform identity disclosures into explanatory narratives. Specifically, when a person merely mentions their identity, the model automatically fills in a causal chain: because you are a member of a certain group, you have likely experienced discrimination, exclusion, or structural pressure, and therefore your suffering has a "reasonable" source.
The problem is that disclosing identity does not mean identity is the cause of suffering. A person's depression may stem from work stress, intimate relationships, or physiological factors, with no relation to identity. But the model, without supporting evidence, treats identity as the default framework for explaining distress—this constitutes a misreading of individual experience. The study conceptualizes this phenomenon as "compassionate othering": the response appears full of empathy, but in reality imposes identity-related assumptions onto the individual's suffering.
The insidiousness of this harm lies precisely here—it does not appear in the form of coldness or rejection, but in the guise of care. The person receives a warm passage of text, but that text simultaneously tells them: your suffering is related to the group you belong to.
Othering is a core concept in social theory and postcolonial studies, referring to the process of marking a group as "the other" that is "different from us," and thereby essentializing explanations of their behavior, circumstances, and suffering. It is not necessarily accompanied by hostility and often appears in the form of goodwill, curiosity, or "understanding." The "compassionate othering" proposed by the study is precisely an extension of this concept in AI interaction: the model, with a caring posture, reframes the person's experience as "the experience of a certain type of person," inadvertently erasing the uniqueness of individual experience. Understanding this concept helps grasp why such harm is difficult to detect through conventional bias detection—it does not manifest as negative emotions or discriminatory wording, but as over-interpretation and attribution substitution.
Marginalized Identities Are More Likely to Be "Over-Interpreted"
The study particularly notes that this identity attribution occurs more frequently when marginalized identities are disclosed. In other words, the model is more inclined to give "identity-based" explanations for the suffering of certain groups, and less so for mainstream identities.
This leads to a concerning consequence: if every time a marginalized group member confides, the model steers toward a narrative of "your identity makes you suffer," then the individual's specific circumstances are overshadowed by the group label. The person may have wanted to discuss their insomnia, unemployment, or breakup, but receives a general discourse about their identity predicament. This may not only deviate from their actual needs but also reinforce an implication—your suffering is identity-determined, structural, and difficult to change.
Marginalized identities refer to group identities that have long been in a position of power disadvantage in the social structure and are vulnerable to discrimination and structural pressure, such as specific races, sexual minorities, people with disabilities, religious minority groups, etc. These identities often co-occur with high frequency alongside narratives of discrimination, poverty, and mental health problems in training data, so models may learn a statistical association and invoke this association as default background knowledge when generating responses. This is precisely a possible mechanism for identity attribution bias: it is not that the model "has bias," but that it generalizes the group-adversity co-occurrence patterns in the corpus to every specific individual, ignoring the actual content of the individual's disclosure.
The Shell of Empathy, the Core of Assumption
The study's wording is worth pondering: these assumptions "frequently appear in otherwise empathetic responses." That is, the model does not exhibit obvious bias or hostility; it is still comforting, listening, and offering advice. The problem lies in the underlying logic of the comfort—it completes the attribution on behalf of the person without supporting evidence.
From a user experience perspective, this is harder to identify than direct coldness. Users find it difficult to question a passage that expresses care for them, but it is precisely this subtle attribution shift that may cause the model's support to lose its specificity. Truly context-sensitive support should be based on what the person themselves discloses, rather than using identity labels to fill causal gaps.
How to Mitigate: Ground Explanations in the Person's Disclosure
The direction given by the study is clear: the key to mitigating this type of harm is to ground explanations of suffering in the individual's own disclosure and avoid unsupported identity-related assumptions.
For model developers, this means incorporating audit procedures targeting identity attribution into training and evaluation, especially checking whether the model proactively constructs identity-suffering causal chains in the absence of evidence. For users, this study also provides a practical reminder: when you confide in an LLM, if it starts attributing your suffering to your identity and you yourself do not think so, then the empathy in that response may be built on a false premise.
Empathy should not come at the cost of defining suffering for others. Models can acknowledge the pressure that identity may bring, but should not impose this possibility as a default explanation on every person who discloses their identity.
One-line Summary
Audit finds that LLMs automatically rewrite confiders' identity disclosures into causes of suffering, forming insidious compassionate othering.
Paragraph Summary
This study systematically audited how six LLMs handle identity disclosures in suicidal ideation and psychological distress scenarios through paired counterfactual interventions. Results show that models tend to transform identity information such as gender, race, and sexual orientation into causal narratives explaining suffering, attributing distress to imagined identity-related adversity, and this attribution is often wrapped in seemingly warm empathic responses, conceptualized as "compassionate othering." When marginalized identities are disclosed, this over-interpretation is more frequent and may overshadow the individual's true circumstances. The study points out that the mitigation direction is to ground explanations of suffering in the person's own disclosure and avoid unsupported identity-related assumptions.
Related articles

Gemini 4 Neon Leak: A Potential Strong Contender Against Opus 5.5
A Hacker News thread suggests Gemini 4 Neon looks like an Opus 5.5 competitor. This article unpacks the product positioning behind this community signal, shifts in the top-model competitive landscape, and what it means for developers' model selection strategy.

Claude Fable 5.2's Self-Authored Short Film 'Fable' Sparks Discussion
A short film 'Fable,' credited to Claude Fable 5.2 for script, visuals, and directing, appears on Hacker News. This article analyzes the significance of full-pipeline AI creation, community reactions, and what it means for content creators.

Anthropic IPO Pricing: A Signal of AI Giants Going Public
Anthropic's IPO pricing draws attention. This article analyzes its potential impact on AI industry valuation, commercialization paths, and the competitive landscape, and highlights public-market expectations and challenges for AI companies.