AI Safety in Multi-Turn Conversations: How Gradually Emerging Clues Get Overlooked by Models

Study reveals: when risk clues are scattered across multi-turn conversations, mainstream AI models' safety compliance plummets from 26%-59% to 3%-6%.
A study by a developer team and community mental health support workers tested six mainstream AI models' safety protections in multi-turn conversations. Nine scenarios involving depression, psychotic symptoms, and domestic violence were designed, each with five clues presented either concentrated or distributed. Results show compliance of 26%-59% when clues are concentrated, dropping to 3%-6% when distributed; Claude Sonnet 5 and Opus 5 performed more consistently with 59%-87% under distributed conditions, though Opus 5 still missed the gold standard in 13% of conversations. The study notes that evaluation paradigms using single-message concentrated risk signals systematically overestimate model protection levels, and developers need to build multi-turn, distributed-clue evaluation sets around real communication patterns with domain expert participation in gold standard development.
When Signals for Help No Longer Appear All at Once
When people confide emotional distress to AI assistants, they rarely lay out the entire problem in one go. More often, signs of depression, fragments of delusions, or hints of domestic violence are scattered across several seemingly ordinary conversations. Yet current mainstream safety evaluations almost all test models using a "single message with concentrated presentation" approach—a systematic deviation from how communication actually happens in the real world.
A study on multi-turn conversation safety protections targets exactly this gap. Researchers collaborated with three community mental health support workers to design nine scenarios in which users share clues related to depression, psychotic symptoms, or domestic violence during a conversation. For each scenario, the support workers pre-established "gold standard" responses: ranging from providing help-seeking information to pausing routine tasks and prioritizing safety concerns.

Experimental Design: Concentrated vs. Distributed Presentation
The key variable in the study was how clues were presented. Each scenario contained five clues, and the experiment set up two conditions: one where the five clues were concentrated in a single message, and another where they were distributed throughout the conversation, interspersed among ordinary topics.
Subsequently, researchers evaluated six widely used models from five developers across more than 7,400 conversations. The criterion was not whether the model "seemed caring," but whether its response actually met the gold standard set by support workers—whether it identified the risk, appropriately guided toward professional support, and interrupted the original task to address safety concerns when necessary.
This evaluation approach, anchored in domain expert standards, is closer to actual protection needs than general safety benchmarks, and it makes the gaps between different models quantifiable.
Gold Standard in this type of evaluation specifically refers to the ideal response pre-determined by domain experts that serves as a criterion for judgment. Unlike approaches relying on model self-assessment or general rules, the gold standard anchors "what constitutes an appropriate response" in the professional judgment of frontline practitioners, giving evaluation results a comparable reference frame. In this study, community mental health support workers established graded gold standards for each scenario: from providing help resources and gently guiding toward professional support, to pausing routine tasks and prioritizing safety concerns when risk escalates. The value of this approach lies in transforming "safety" from a vague impression into operable judgment criteria, and enabling gaps between different models to be quantified. The quality of the gold standard depends heavily on the professional experience of its creators, making collaborative design a key prerequisite for the credibility of such evaluations.
The Gap Revealed by Data: From 26% to 3%
The results show a stark divergence. When clues appeared in concentrated form, GPT-5.5, Grok 4.3, DeepSeekAd V4, and Llama 4 Maverick met support workers' standards in only 26% to 59% of conversations; when clues were distributed throughout the conversation, this proportion plummeted to 3% to 6%.
Claude Sonnet 5 and Claude Opus 5 performed noticeably more consistently: 68% to 97% compliance with concentrated presentation, and still 59% to 87% with distributed presentation. But even the best-performing Claude Opus 5 failed to meet the gold standard in 13% of conversations under distributed clue conditions.
What these numbers reveal is not just that "some models are safer," but that the entire evaluation paradigm may overestimate protection levels. Developer-reported improvements are likely built on test sets where "signals are sufficiently clear and concentrated"—yet real users simply don't express themselves that way.
Why Distributed Clues Are Harder to Identify
With concentrated presentation, five clues form a strong cluster of risk signals, and models easily categorize it as a conversation requiring intervention. Once distributed, each clue may appear mundane on its own—a complaint about sleep, a vague suspicion about a colleague, a casual mention of a family argument. The model needs to actively connect these fragments across multiple turns and judge their cumulative meaning.
This requires the model to have cross-turn intent tracking capability, as well as the ability to weigh "completing the user's current task" against "prioritizing safety risks." The support workers in the study explicitly stated that in certain situations the correct approach is to pause routine tasks and directly address safety concerns—which creates tension with the "responsive to every request" tendency that models are trained toward.
Cross-turn intent tracking refers to the model's ability to continuously maintain and update internal representations of user intent, emotional state, and risk clues across multiple turns of conversation. Single-turn evaluation only requires the model to react to the current message, while multi-turn scenarios require it to associate fragmentary information that appeared earlier with the current message and judge whether their cumulative meaning constitutes a risk signal. This capability faces two challenges: first, information within the context window may be diluted by subsequent ordinary topics, and the model tends to "forget" earlier weak clues; second, models are typically optimized to focus on completing the current request, lacking training signals to actively retroactively integrate historical information. Therefore, distributed clue scenarios essentially test whether the model can continuously track and update risk states beyond task execution.
Implications for Developers and Evaluators
The study's core conclusion points directly at evaluation methodology: safety evaluations centered on clear, easily readable concerns will overestimate the degree to which protective measures safeguard people when distress signals emerge gradually or indirectly. If developers want their claims of "safety improvements" to be taken seriously, they must design and evaluate protective mechanisms around how people actually communicate.
This means at least two things. First, safety test sets need to incorporate multi-turn, distributed, low-intensity clue distributions, rather than only testing explicit crisis expressions. Second, evaluation standards should be developed with domain expert participation, rather than relying solely on the model itself or general rules to judge what constitutes an "appropriate response."
For users who rely on AI assistants for emotional support, this study also offers a practical reminder: a model's considerate response in a single conversation does not mean it can consistently identify risks over long-term interaction. When distress unfolds gradually, actively and clearly expressing needs remains important.
External validity is a metric for whether research conclusions can be generalized to real-world situations. In this study, external validity is reflected in how closely the experimental scenarios match real users' ways of expression: distributed clues of varying intensity, interspersed among everyday topics, are precisely the typical form people use when confiding distress. Having community mental health support workers participate in designing scenarios and gold standards enhanced this fidelity, because frontline practitioners are familiar with how risk signals gradually emerge in real conversations. In contrast, test sets constructed solely by developers tend to favor clear, concentrated crisis expressions, and their conclusions may be distorted when generalized to real users. When evaluating multi-turn safety protections, external validity is as important as internal reproducibility—the former determines whether claims of "safety improvements" can withstand real-world testing.
Consensus and Divergence Across Multiple Perspectives
The study was completed through collaboration between a developer team and community mental health support workers, with experimental design and gold standards coming from frontline practitioners, which provides strong external validity for its conclusions. Industry observers widely agree with the judgment that "concentrated clue testing overestimates safety," but diverge on where responsibility lies: one side believes model developers should build multi-turn risk tracking capabilities into training and evaluation processes; the other emphasizes that AI assistants should not be treated by default as mental health support tools, and that product design needs clearer boundaries and referral mechanisms.
Regardless of position, one consensus is clear: the effectiveness of safety protections must ultimately be tested in how people actually speak, not in conversations conveniently constructed by evaluators.
Related articles

Harvard AI Safety Course CS 2881R Sparks Discussion
Harvard's CS 2881R AI Safety course was mentioned on Hacker News. This article covers the course background, core AI safety topics, and takeaways for learners, helping readers understand how top universities cultivate AI safety talent.

Expanding Your Social Circle: Friend-Making Apps Worth Trying
How to choose a friend-making app? This article reviews friendship apps like the friend-discovery platform BFF and activity-oriented Timeleft, analyzing which approach suits whom and offering practical tips on offline meeting safety and expectation management.

Why You Shouldn't Run the Hermes Agent on Your Personal Computer
Why shouldn't you run autonomous AI agents like Hermes on your personal computer? This article analyzes why agents should be deployed to a dedicated VPS instead of your primary device, from four angles: security risks, resource consumption, credential leakage, and autonomy requirements—plus the practical logic of isolated deployment.