Latent Undertow
An academic paper investigating the robustness of LLM safety probes against input perturbations. It demonstrates that minor perturbations such as spelling errors can cause safety probes to misclassify inputs severely. The paper introduces a geometric pattern termed 'rotation-decay' and proposes a KV-cache forking defense mechanism to mitigate such vulnerabilities.
Timeline (last 90 days)
The study "Latent Undertow" found that ordinary spelling errors in the input are sufficient to cause severe misjudgments in safety probes that read the hidden states of the model.
When the input contains spelling errors, the vector read by the probe at the perturbed token position undergoes a rotation of 43 to 56 degrees.
The effect of the perturbation drops below 15% after approximately 10 downstream tokens (rapid decay).
Stacking approximately 3 common spelling errors in a single message causes the true positive rate of a single-position prompt injection probe to drop by 12 percentage points at FPR=1%.
The study proposes a KV-cache fork method for single-position probes, which allows the probe to read hidden states at downstream positions after perturbation by appending a fixed short suffix after the user message.
All Facts (5)
The study "Latent Undertow" found that ordinary spelling errors in the input are sufficient to cause severe misjudgments in safety probes that read the hidden states of the model.
50%UnverifiedWhen the input contains spelling errors, the vector read by the probe at the perturbed token position undergoes a rotation of 43 to 56 degrees.
50%UnverifiedThe effect of the perturbation drops below 15% after approximately 10 downstream tokens (rapid decay).
50%UnverifiedStacking approximately 3 common spelling errors in a single message causes the true positive rate of a single-position prompt injection probe to drop by 12 percentage points at FPR=1%.
50%UnverifiedThe study proposes a KV-cache fork method for single-position probes, which allows the probe to read hidden states at downstream positions after perturbation by appending a fixed short suffix after the user message.
50%