How Ordinary Typos Can Fool LLM Safety Probes: A Deep Dive into the Latent Undertow Research

An Overlooked Security Blind Spot

Large language models (LLMs) are generally quite tolerant of everyday input variations — a typo here, a missing punctuation mark there rarely changes the user's intent or meaningfully affects the model's response. But a new study titled Latent Undertow has uncovered a counterintuitive phenomenon: these seemingly harmless perturbations are enough to cause serious misfires in safety probes that read the model's hidden states.

A probe, in this context, is a defense mechanism that detects malicious prompts (such as prompt injection attacks) by reading vectors from the model's internal hidden layers. Rather than relying on model outputs, it directly "sees through" the model's internal representations. This approach has been widely regarded as an important line of defense against prompt injection — but this research shows that its robustness to input perturbations is far more fragile than previously assumed.

Prompt injection attacks occur when an attacker embeds malicious instructions within an input to hijack or deceive an LLM into executing unintended behavior. Much like traditional SQL injection, the attacker exploits the model's blurry boundary between instructions and data, slipping disguised malicious commands into the model's processing pipeline. Safety probes serve as a countermeasure based on the idea that malicious prompts activate distinctive hidden-state patterns inside the model — a lightweight linear classifier trained to recognize these patterns can intercept attacks before the model generates any output. The appeal of this approach is its independence from model outputs, making it theoretically resistant to purely linguistic obfuscation. Yet Latent Undertow directly challenges this assumption of "independence."

Rotation and Decay: The Geometry of Perturbation

The study's central finding can be summarized as a "rotation-and-decay" geometric pattern. When a typo appears in the input, the vector read by the probe at the perturbed token position rotates by 43 to 56 degrees — a substantial shift. Interestingly, however, this shift decays rapidly: roughly 10 tokens downstream, the perturbation's influence falls below 15%.

In other words, the perturbation moves through the model's latent space like an undertow beneath the water's surface (hence the paper's title, Latent Undertow) — stirring up violent local turbulence, yet quickly settling as it propagates. This geometric pattern is not an anomaly specific to one model: the researchers reproduced it across three distinct models — Llama-3.1-8B, Qwen3-8B, and Gemma-4-E4B — suggesting it is a systemic property of current mainstream architectures, not a one-off quirk.

Understanding the "rotation" metaphor requires some background. An LLM's hidden layers represent each token as a vector in a high-dimensional space, and a probe essentially defines a decision hyperplane in that space to distinguish "malicious" from "benign" representations. When a token changes (e.g., due to a typo), the direction of the vector the model generates at that position shifts — this is the "rotation." A 43–56 degree rotation means the cosine similarity between the original and perturbed vectors falls to roughly 0.6–0.7, which is more than enough for a linear probe to push a vector that should land on the "malicious" side of the decision boundary into the "benign" region, causing a miss. The "decay" effect, meanwhile, stems from the Transformer's self-attention mechanism: as the sequence continues, the model integrates context across multiple heads, and the influence of a local perturbation gets "diluted" by subsequent token representations, quickly reverting toward the original direction.

Extremely Low Attack Cost, No Simple Fix

What makes this vulnerability particularly threatening in practice is how cheap it is to exploit. The study shows that stacking just ~3 common typos in a single message causes the true positive rate (TPR) of a single-position prompt injection probe to drop by 12 percentage points at FPR=1%. This means attackers don't need to craft sophisticated adversarial examples — a few seemingly accidental typos are enough to significantly degrade detection capability.

What's more, this performance gap cannot be closed simply through recalibration. The study also evaluated perturbation-augmented training, a common remediation approach, and found it only narrows the gap to -3.7 percentage points — a limited improvement.

Multi-Position Aggregation: A Partial Fix

One natural improvement is multi-position aggregation — having the probe synthesize information from multiple token positions rather than reading a single point. The study found this works well against localized perturbations (loss ≤ 0.5 percentage points), essentially "curing" perturbations concentrated at a single location.

However, against distributed perturbations — where perturbations are spread across multiple positions — multi-position aggregation only partially helps. Even with attention-based or max-based aggregators, performance still drops by roughly 3.8 percentage points.

KV-Cache Forking: An Elegant Engineering Solution

The study's most valuable contribution is the KV-cache fork method, proposed specifically for single-position probes. It directly leverages the "fast spatial decay" finding: since the perturbation's influence fades within a few downstream tokens, simply append a short fixed suffix after the user's message and have the probe read the hidden states a few tokens downstream from the perturbed position — sidestepping the region of maximum disruption.

The results are striking: this method closes 95% of the performance gap, leaving a residual of just -0.6 percentage points — an order of magnitude better than perturbation-augmented training (-3.7pp). This underscores an important insight: understanding the geometric mechanism behind a vulnerability often leads to more targeted solutions than blindly augmenting training data.

KV-cache (key-value cache) is a standard acceleration technique in Transformer inference: as the model generates tokens one by one, it caches the Key and Value matrices from each layer for all previously processed tokens to avoid redundant computation. KV-cache forking exploits this mechanism directly — when a short fixed suffix (such as a newline or special token) is appended to the end of the user's message, the model continues computing hidden states for the suffix positions on top of the existing cache, and these positions have already "digested" the upstream perturbation through the attention mechanism. Since only a tiny number of extra suffix tokens are needed, the computational overhead is nearly negligible — no model weight changes required, no probe retraining needed. It's essentially a zero-cost engineering patch. The elegance of this approach lies in the fact that it doesn't fight against the perturbation; instead, it works with the model's own information-integration mechanism to naturally "heal" the local noise introduced by the perturbation.

Implications for AI Security Practice

This research serves as a reminder that while hidden-state probes are powerful, their robustness may be significantly underestimated. Attackers can bypass detection using natural perturbations at near-zero cost — perturbations that look completely harmless to human reviewers and are nearly impossible to catch with content moderation alone.

For teams deploying prompt injection defenses, several practical takeaways stand out: first, don't assume probes are inherently robust to input noise — explicitly evaluate performance under spelling perturbations; second, lightweight methods like KV-cache forking that leverage the model's internal geometric properties may be more efficient than retraining; and third, distributed perturbations remain an open and unsolved challenge. The research team has open-sourced the code on GitHub (eladd-ai/latent-undertow), providing a foundation for follow-up research and real-world validation.