Hypersensitive Safety Head
A subset of sparse attention heads in the Transformer attention mechanism that become highly specialized following safety alignment training. These heads produce anomalous activations in response to ambiguous but benign prompts, leading to excessive refusal behavior.
Timeline (last 90 days)
The paper finds the existence of a small number of hypersensitive safety heads, which are a sparse subset of Transformer attention mechanism heads that are mistakenly triggered when processing hard-safe prompts.
Hypersensitive safety heads exhibit abnormal attention entanglement, forcibly binding target entities that are in fact harmless to refusal semantics.
SRC precisely locates hyper-sensitive safety heads during the inference stage and dynamically suppresses them.
All Facts (3)
The paper finds the existence of a small number of hypersensitive safety heads, which are a sparse subset of Transformer attention mechanism heads that are mistakenly triggered when processing hard-safe prompts.
50%UnverifiedHypersensitive safety heads exhibit abnormal attention entanglement, forcibly binding target entities that are in fact harmless to refusal semantics.
50%UnverifiedSRC precisely locates hyper-sensitive safety heads during the inference stage and dynamically suppresses them.
50%