Explainable Suicide Risk Assessment: From Prediction to Evidence Grounding

Why Risk Prediction Alone Isn't Enough

Detecting suicide risk signals from social media posts has long been a key area of research at the intersection of mental health and AI. The massive volumes of text on social network services (SNS) contain a wealth of psychological crisis signals, and identifying them in a timely manner is crucial for early intervention. Yet the dominant approach today often stops at "risk classification" — determining whether a given post falls into a high-risk category — and this approach has clear limitations.

A pure risk-level judgment functions like a black box: it delivers a verdict but cannot explain its reasoning, nor can it illuminate the psychosocial factors underlying the risk. For clinicians and intervention workers, knowing why is often more important than knowing what. When a post is flagged as high-risk, which specific words triggered the judgment? Does it reflect loneliness, loss, interpersonal conflict, or some other deeper cause? That kind of information is what actually enables intervention.

Drawing on the IEEE BigData 2026 Explainable Suicide Risk Detection Challenge, this research proposes a comprehensive framework aimed at moving risk assessment beyond "delivering verdicts" toward "presenting evidence and analyzing contributing factors."

A Three-Part Framework

The framework consists of three interrelated task modules: Risk Assessment, Evidence Grounding, and Factor Identification. This design means that system outputs are no longer isolated risk labels, but a complete analytical chain that can be examined and understood.

Risk Assessment: Routing by Length

Social media posts vary enormously in length — from a few words to lengthy personal narratives. The Risk Assessment module introduces a length-based routing mechanism that applies tailored processing strategies to posts of different lengths. This design directly addresses the heterogeneity of real-world data: short and long texts differ substantially in semantic density and contextual richness, and applying a single unified model to both tends to underperform on each.

This module achieved a Weighted F1 score of 0.8088, demonstrating reliable performance in risk-level classification.

What is Weighted F1? Weighted F1 is a widely used aggregate metric in multi-class evaluation. It computes the F1 score for each class and takes a weighted average based on each class's sample size in the dataset, giving a result that better reflects overall performance when class distributions are imbalanced. By contrast, Macro F1 assigns equal weight to every class and is more sensitive to rare categories. In suicide risk datasets, high-risk samples are typically far fewer than low-risk or no-risk samples, so both metrics carry meaning: Weighted F1 reflects the system's overall reliability, while Macro F1 better reveals the system's ability to detect minority classes (such as high-risk individuals).

Evidence Grounding: Making Judgments Accountable

The Evidence Grounding module is responsible for locating the key phrases in the original text that support the risk judgment. More importantly, the research introduces a Risk-Evidence constraint to ensure that the extracted evidence remains consistent with the risk prediction. This addresses a common pitfall in explainability: the "explanations" a model provides sometimes diverge from its actual decision logic. Through this constraint mechanism, the textual evidence the system extracts genuinely corresponds to the basis for its risk judgment.

Evidence Grounding achieved a Macro row F1 of 0.7605 on the test set, indicating that it can fairly accurately annotate the specific text spans that trigger risk signals.

The faithfulness problem in Explainable AI (XAI): A long-standing challenge in XAI is faithfulness — the explanations a model generates may not truly reflect its decision process, but rather serve as post-hoc rationalizations. In high-stakes applications, this is especially dangerous: a seemingly reasonable explanation can mask flawed decision logic. The Risk-Evidence constraint introduced in this research is essentially a consistency constraint, requiring that the extracted evidence text must logically support the risk prediction label. This partially addresses the faithfulness problem at an architectural level, rather than relying solely on a separate post-hoc explanation module.

Dual Verifiers Tackle Factor Identification

Factor Identification is the most challenging of the three tasks, with the goal of identifying the fine-grained psychosocial factors underlying a post. The research employs a strategy of two verifiers working in tandem.

Taxonomy Verifier and Evidence-Aware Verifier

The first is the Taxonomy Verifier, which focuses on the semantic characteristics of factors themselves, making judgments based on a predefined factor classification taxonomy. The second is the Evidence-Aware Verifier, which leverages factor-specific lexical-semantic cues to select highly informative positive training examples.

The two verifiers have complementary strengths: the first emphasizes accuracy in semantic categorization, while the second focuses on capturing discriminative signals from textual evidence. The system ultimately fuses the predicted probabilities from both verifiers to produce the final factor judgment. This ensemble strategy partially compensates for the blind spots of either method alone.

That said, the Factor Identification module achieved a Macro F1 of only 0.5562, noticeably lower than the other two tasks. This reflects the inherent complexity of identifying psychosocial factors — where boundaries between factors are blurry, sample distributions are uneven, and semantic overlap is severe — challenges that remain unsolved in this field.

What are psychosocial factors? Psychosocial factors are composite social and psychological influences on an individual's mental state and behavior. In suicide risk assessment, they typically encompass dimensions such as interpersonal conflict, unemployment or financial stress, bereavement, loneliness, substance abuse, and prior trauma. In text, these factors are often expressed in subtle, indirect ways, and they overlap heavily with one another semantically — for example, "loneliness" and "interpersonal conflict" may share many similar surface-level expressions in discourse. This is the core difficulty of factor identification: it demands fine-grained semantic discrimination rather than simple keyword matching, while also contending with severe class imbalance across factor categories.

Toward Explainable Mental Health AI

The core value of this work lies in advancing suicide risk assessment beyond pure "risk level prediction" to a more interpretable level of analysis. The system doesn't just tell you whether a post is high-risk — it can pinpoint the specific textual evidence supporting that judgment and provide fine-grained information about the underlying psychosocial factors.

Looking at the F1 scores across all three tasks, Risk Assessment (0.8088) and Evidence Grounding (0.7605) have reached a level that is reasonably practical, while Factor Identification (0.5562) reveals the direction for future improvement. For real-world applications, this kind of explainability is especially valuable — it enables AI judgments to be reviewed by human experts and better meets the stringent demands of mental health intervention for transparency and trustworthiness.

In high-stakes, high-sensitivity domains like mental health, explainability is not a nice-to-have — it is a prerequisite for deployment. A system that cannot explain its reasoning will struggle to be genuinely integrated into clinical or intervention workflows. The framework paradigm this research proposes — a unified triad of risk, evidence, and factors — offers a valuable reference path for building trustworthy AI tools for mental health.