ShallowDefend
A multi-turn dialogue safety defense framework based on shallow alignment properties. It intervenes at the front end of output through explicit safety decision tokens and comprises a three-step process of evaluation, decision and response.
Timeline (last 90 days)
ShallowDefend proposes the Assess → Decide → Respond three-stage framework: assess user intent, make an explicit parseable safety decision, and generate the response based on the decision.
In the ShallowDefend framework, the safety decision is represented as a token segment, and the model must first output an explicit safety judgment before continuing generation.
ShallowDefend supports self-verification rewards (rule-based, requiring no additional judge model or human annotation) and LLM-as-judge rewards (used for the graded decision space).
ShallowDefend training takes place at the "payload turn", the turn in which the harmful intent actually surfaces, and uses reinforcement learning to consolidate the decision behavior.
ShallowDefend covers two model families and fourteen benchmarks, and is evaluated on the three dimensions of safety, utility and helpfulness.
In controlled comparisons, ShallowDefend achieves the strongest multi-turn safety performance and maintains utility, but pays a certain cost in helpfulness.
Token-by-token KL analysis shows that the effect of ShallowDefend training is concentrated in the brief deliberation interval at the start of the response.
Interventional probes show that the decision committed during the deliberation interval constitutes a causal control point—change it, and the direction of the subsequent response changes accordingly.
ShallowDefend suggests a shift in thinking: rather than doing complex filtering and rewriting at the response level, implant a verifiable, trainable explicit decision at the very front of model generation.
Deploying ShallowDefend requires weighing a moderate decline in helpfulness and dependence on a decision-label system.
All Facts (10)
ShallowDefend proposes the Assess → Decide → Respond three-stage framework: assess user intent, make an explicit parseable safety decision, and generate the response based on the decision.
50%UnverifiedIn the ShallowDefend framework, the safety decision is represented as a token segment, and the model must first output an explicit safety judgment before continuing generation.
50%UnverifiedShallowDefend supports self-verification rewards (rule-based, requiring no additional judge model or human annotation) and LLM-as-judge rewards (used for the graded decision space).
50%UnverifiedShallowDefend training takes place at the "payload turn", the turn in which the harmful intent actually surfaces, and uses reinforcement learning to consolidate the decision behavior.
50%UnverifiedShallowDefend covers two model families and fourteen benchmarks, and is evaluated on the three dimensions of safety, utility and helpfulness.
50%UnverifiedIn controlled comparisons, ShallowDefend achieves the strongest multi-turn safety performance and maintains utility, but pays a certain cost in helpfulness.
50%UnverifiedToken-by-token KL analysis shows that the effect of ShallowDefend training is concentrated in the brief deliberation interval at the start of the response.
50%UnverifiedInterventional probes show that the decision committed during the deliberation interval constitutes a causal control point—change it, and the direction of the subsequent response changes accordingly.
50%UnverifiedShallowDefend suggests a shift in thinking: rather than doing complex filtering and rewriting at the response level, implant a verifiable, trainable explicit decision at the very front of model generation.
50%UnverifiedDeploying ShallowDefend requires weighing a moderate decline in helpfulness and dependence on a decision-label system.
50%