Shallow Alignment Becomes a Defense Strength: How ShallowDefend Safeguards Multi-Turn Conversations

Large language model safety alignment performs reliably in single-turn conversations, but once the same harmful request is split across multiple turns, the success rate rises noticeably. This fragility is no accident—it stems from the "shallow" nature of alignment itself. ShallowDefend's research offers a counterintuitive solution: since safety behavior is concentrated in the first few tokens of output, let the defender seize that high-leverage window first.

Why Multi-Turn Conversations Can Break Through Safety Alignment

The research points out that the mechanism behind existing safety alignment is fundamentally "shallow": the model's safety behavior is highly concentrated in the first few tokens of its response. When a harmful request is made directly, the model can often trigger a refusal within those few tokens; but when the same request is decomposed and advanced gradually across multiple turns, this early-trigger mechanism is easily bypassed.

More critically, existing defenses inherit this weakness. They supervise the surface form of responses but do not give the model an explicit safety decision to learn. What the model learns is "saying it this way looks safe," not "I judge this to be harmful, therefore I refuse." The former easily fails under multi-turn inducement.

Safety Alignment generally refers to using supervised fine-tuning, reinforcement learning from human feedback (RLHF), or direct preference optimization (DPO) to make large language models refuse harmful requests and follow safety norms. Its training signal is often concentrated at the beginning of responses, because the intent to refuse or comply is already determined in the first few tokens of generation. This "shallow" characteristic makes models robust under direct single-turn requests but ill-equipped against attacks that gradually induce and distract across multiple turns. Understanding this helps clarify why ShallowDefend chooses to intervene at the front end of output rather than the back end.

Assess → Decide → Respond: Making the Decision Explicit

The framework proposed by ShallowDefend has three steps: assess user intent, make an explicit and parseable safety decision, then generate a response based on that decision. Unlike previous approaches that directly supervise response content, the key here is the middle "decision" step—it is represented as a token segment, and the model must first output an explicit safety judgment before continuing generation.

The advantage of this design lies in trainability. Since the decision itself is a sequence of tokens, rewards can be implemented with rules: self-verification rewards compare the label submitted by the model against known properties of the conversation, requiring neither an additional judge model nor human annotation. The same format also supports LLM-as-judge rewards over a graded decision space, combined with safety redirection, allowing the model to offer constructive alternatives beyond refusal.

Training occurs at the "payload turn"—the turn where the harmful intent actually manifests—using reinforcement learning to solidify the decision behavior.

LLM-as-judge is a method that uses a large language model itself as an evaluator: given the conversation context and the safety decision label submitted by the model, the judge model determines whether the decision is correct according to predefined criteria and outputs a reward signal. Compared with rule-based self-verification rewards, LLM-as-judge can handle finer-grained graded decision spaces (such as "safe," "borderline," and "harmful"), but at higher computational cost. ShallowDefend accommodates both reward types under the same token format, enabling low-cost training of binary decisions while also scaling to more complex safety-level judgments when needed.

Experimental Performance: Leading Safety with Controllable Utility Cost

The research covers two model families and fourteen benchmarks across three dimensions: safety, utility, and helpfulness. In controlled comparisons, ShallowDefend achieved the strongest multi-turn safety performance while maintaining safety leadership and preserving utility, paying a certain cost only in helpfulness.

This trade-off result is noteworthy. Many safety defense schemes either sacrifice model capability substantially or fail to defend in multi-turn scenarios. ShallowDefend's positioning is "no regression in safety, utility largely preserved, moderate concession in helpfulness"—which is more acceptable in real-world deployment.

Mechanism Verification: The Decision Point Is a Causal Control Point

The paper does not stop at metric improvements but verifies the cause through token-by-token KL analysis and interventional probes. Results show that the training effect is concentrated in the brief "deliberation" span that sets the response trajectory, and the decision submitted here constitutes a causal control point—change it, and the direction of the subsequent response changes accordingly.

This explains why the method works: the shallowness exploited by attackers is precisely transformed by defenders into a lever for intervention. Since a short span determines the trajectory of the entire response, guarding this span is equivalent to guarding the entire turn of conversation.

Token-by-token KL analysis measures changes in the model's output distribution at each generation position before and after training. If the KL divergence is significantly concentrated on the first few tokens of the response, it indicates that training mainly changed the model's behavioral tendency in that interval rather than uniformly affecting the entire response. Interventional probes artificially modify the hidden state or token input at a certain position during inference and observe whether subsequent generation changes accordingly, thereby judging whether that position constitutes a causal control point. Combined, the two analyses can distinguish "correlation" from "causation," providing empirical support for the mechanistic explanation of ShallowDefend.

Implications for Safety Alignment Practice

The value of this work goes beyond a single defense framework. It suggests a shift in thinking: rather than doing more complex filtering and rewriting at the response level, it is better to implant a verifiable, trainable explicit decision at the very front end of model generation. The "shallowness" of safety alignment was once seen as a flaw, but from another angle, it is also the position with the lowest intervention cost.

Of course, the moderate decline in helpfulness and the dependence on a decision-label system remain issues to weigh in deployment. But at least methodologically, ShallowDefend offers a self-consistent explanation: shallowness is not a vulnerability to be patched but a strategic position that can be actively occupied.

One-line Summary

ShallowDefend leverages the shallowness of alignment to preemptively implant explicit safety decisions in multi-turn conversations to defend against attacks.

Paragraph Summary

The article explores why large model safety alignment is easily broken in multi-turn conversations, pointing out that the root cause is safety behavior being concentrated in the first few tokens of output, while existing defenses only supervise the surface form of responses without explicitly learning safety decisions. ShallowDefend proposes an Assess→Decide→Respond framework that has the model first output a parseable safety decision token segment, then generate a response based on that decision, using self-verification rule rewards for reinforcement learning training. Experiments cover two model families and fourteen benchmarks, achieving the strongest multi-turn safety performance while preserving utility at only a moderate cost to helpfulness. Token-by-token KL analysis and interventional probes confirm that this decision point is a causal control point, showing that shallowness can be transformed into a defense lever.