Breaking LLM Over-Refusal: A Deep Dive into Semantic Routing Calibration (SRC)

When Safety Alignment Goes Too Far: The LLM Over-Refusal Problem

Large language models (LLMs) trained with safety alignment frequently exhibit a frustrating behavior known as over-refusal. These models incorrectly reject instructions that are completely harmless but happen to touch on safety-adjacent topics on the surface. Ask a model "how to kill a Linux process," for instance, and it may trigger a refusal response just because of the word "kill" — treating a perfectly ordinary technical question as a dangerous request.

This phenomenon directly undermines model usability. Users want AI that is both safe and helpful, yet overly cautious safety guardrails cause models to repeatedly fail on everyday tasks. A recent paper published on arXiv (arXiv:2609.25049) offers a new mechanistic explanation for this problem — and a corresponding solution.

Why Previous Explanations Fall Short: From Static to Dynamic

Prior research largely attributed over-refusal to static representation overlap — the idea that benign and harmful prompts cluster too closely together in the model's internal embedding space, making them hard to distinguish. While intuitive, this explanation overlooks the underlying dynamic mechanisms at play.

The paper's authors argue that a purely static perspective fails to get at the root cause of over-refusal. What actually happens inside a model during inference that causes it to incorrectly reject a "Hard-Safe" prompt — one that is genuinely harmless but sits in an ambiguous, safety-adjacent region? Answering this requires looking inside the Transformer's attention mechanism itself.

The concept of static representation overlap comes from analyzing a model's internal embedding space. In vector space, semantically similar concepts cluster together, and the essence of safety alignment training is to teach the model to identify "harmful regions" and refuse inputs that fall within them. The problem is that some benign prompts — such as technical instructions involving "killing a process" — happen to sit close to genuinely harmful prompts in semantic embedding space, creating so-called representation overlap. Traditional improvement strategies follow from this view, attempting to push the two clusters apart through better training data or contrastive learning. But this static perspective is fundamentally analyzing a snapshot of the result after inference, not the inference process itself. In a Transformer model generating tokens one by one, information flows and recombines dynamically across multiple attention layers. The final output is the product of a complex series of computational interactions, not simply determined by initial embedding distances. Focusing solely on distances in representation space is like reading only the final score of a game while ignoring everything that happened during it.

The Core Finding: "False Triggers" in Hypersensitive Safety Heads

The paper's key insight is the discovery of a small set of hypersensitive safety heads — a sparse subset of Transformer attention heads that "false-trigger" when processing Hard-Safe prompts.

Specifically, these attention heads exhibit abnormal attention entanglement: they forcibly bind otherwise harmless target entities to rejection semantics. For example, when a user's query contains a neutral noun, these safety heads artificially associate that noun with a "should refuse" signal.

This binding triggers a severe high-entropy routing conflict. The consequence is that the target entity that actually deserves attention gets starved of the attention it needs, the model's reasoning pathway becomes distorted, and the result is an incorrect refusal output.

Understanding this finding requires a basic grasp of Transformer attention. In a Transformer architecture, each layer contains multiple attention heads, each of which can independently learn to attend to different positions and types of relationships in the input sequence. Research has shown that different attention heads tend to serve different semantic functions: some capture syntactic structure, some track entity coreference, and some relate to domain-specific knowledge. "Safety heads" are those that, through safety alignment training, have gradually specialized in detecting signals of harmful content. Under normal circumstances, they should only activate when genuinely harmful content is present. High-entropy routing conflict describes a pathological state of the attention distribution: entropy measures the uncertainty of a probability distribution, and high entropy means attention is diffusely spread across many positions rather than concentrated on the key semantic entities. When safety heads abnormally activate and "hijack" a portion of the attention weights, the model's information routing becomes disordered, critical semantic signals cannot propagate effectively to subsequent layers, and the output layer is likely to generate a refusal response.

The SRC Framework: A Lightweight, Training-Free Inference-Time Solution

In response to this mechanism, the paper proposes the Semantic Routing Calibration (SRC) framework. It offers several notable engineering advantages:

Training-Free and Lightweight

SRC is a training-free inference framework that can be deployed without retraining or fine-tuning the model. This significantly lowers the barrier to adoption and avoids potential performance regressions that retraining might introduce.

Precise Localization and Dynamic Suppression

The first step of SRC is to precisely localize the hypersensitive safety heads at inference time and dynamically suppress them. By suppressing these false-triggering attention heads, the model is freed from the abnormal attention entanglement.

Dual-Branch Logits Fusion as a Safety Regularizer

Simply suppressing safety heads risks degrading safety performance. To address this, SRC introduces a dual-branch logits fusion mechanism, which acts as a "safety regularizer" during subsequent decoding — restoring normal inference while preserving as much of the model's inherent safety capability as possible. This reflects a design philosophy aimed at balancing usability with safety.

Logits fusion is a well-known class of inference-time techniques for intervening in the output distribution of large language models. When an autoregressive language model generates each token, the final layer outputs raw scores (logits) over the vocabulary, which are normalized via softmax into a probability distribution before sampling or argmax selection. The dual-branch fusion approach works by maintaining two inference branches simultaneously — one where the hypersensitive safety heads have been suppressed (restoring normal semantic routing), and one where the model's original safety activations remain intact. During decoding, the logits from both branches are combined with a weighted average, so the final output benefits both from the routing correction of the suppression intervention and from the refusal tendency instilled by original safety training as a "baseline." This design is conceptually similar to Contrastive Decoding or the multi-path ideas in speculative decoding — in essence, leveraging the complementarity of two signals to achieve finer-grained output control without making any permanent modifications to the model's weights.

Experimental Results and Significance

Extensive experiments in the paper demonstrate that SRC effectively mitigates over-refusal while preserving the model's inherent safety performance to the greatest feasible extent. In other words, it challenges the seemingly forced tradeoff between being overly cautious and being safety-permissive.

From a broader perspective, the value of this work lies not only in proposing a concrete solution, but in advancing the analysis of over-refusal from static representations to dynamic attention mechanisms. This mechanistic interpretability research approach is becoming an increasingly important direction for understanding and improving large model behavior. When we can precisely identify "which attention heads are causing problems," interventions can be made with greater precision and lower cost.

Summary

Over-refusal is a classic side effect of safety alignment. SRC provides a lightweight, training-free solution by identifying hypersensitive safety heads, dynamically suppressing false triggers, and applying dual-branch fusion regularization. For researchers and engineers focused on balancing LLM safety with usability, this attention-level analytical perspective is well worth deeper exploration.

Note: This article is based on the arXiv preprint (arXiv:2609.25049v1). All conclusions are subject to the paper's final published version.