Add a Few Spaces and DeepSeek Falls Apart? ByteDance Paper Reveals the Phase Blind Spots of Chunked KV Compression

Ask DeepSeek the same question twice, just add a few extra spaces, and the answers you get can be one like a "genius" and one like an "idiot." This isn't a random hallucination—it's a precisely reproducible structural flaw that a recent ByteDance team paper has exposed.

The "Dual Nature of Genius and Fool" Triggered by a Single Space

The ByteDance team's discovery began with an utterly ordinary task: having DeepSeek^Ad-V4-Flash-Base complete a piece of code.

Specifically, completing the last token of an FP8 quantization function. According to the code logic, the correct result should be 8. But after the researchers added a string of purely decorative equals signs at the very beginning of the code, the model output 32.

Even more bizarrely, the error wasn't random—it was strictly tied to the number of equals signs—when the number of equals signs modulo 4 was 0 or 1, the error rate was as high as 71.3%; when the remainder was 2 or 3, the accuracy returned to 91.5%.

This phenomenon of "adding a few spaces and it falls apart" was dubbed phase sensitivity by the ByteDance team. The paper is titled Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression, directly calling out the "dual nature of genius and fool" problem in the DeepSeek V4 series models.

Root Cause: The "Position Determines Fate" Brought by Chunked Compression

The Other Side of the Cost-Saving Killer Feature

Chunked KV Cache compression is a core technique DeepSeek uses to address the memory pressure of long contexts. It splits consecutive tokens into fixed-length segments, and the tokens in each segment are compressed into a "small summary" through a learnable gating layer, greatly reducing the memory pressure of long contexts.

This solves the memory problem but introduces a new one.

In a standard Transformer, attention depends only on the relative distance between tokens, not their absolute position—no matter where a sentence is placed, as long as the word order is unchanged, the meaning is unchanged. But with chunked compression, each token's position within the compressed segment becomes crucial; this position is called the "phase." The gating layer assigns different weights to tokens at different positions (slot gain factors): tokens that land in strong slots are weighted and retained, while tokens that land in blind-spot slots are almost ignored.

Delving further into the model's internals, the researchers found that attention heads form a natural division of labor—some specialize in positions near the front of a segment, others in positions near the back. This "phase specialization" makes certain positions systematically weak links.

What is KV Cache? During autoregressive generation in a Transformer model, each time a new token is generated, attention must be computed over the Key and Value vectors of all previous tokens. To avoid redundant computation, inference systems cache these K and V vectors—this is the KV Cache. The problem is that its memory footprint is proportional to context length: the longer the context, the larger the cache. Taking 128K tokens, dozens of layers, and multi-head attention as an example, the KV Cache can easily occupy tens of GB of memory, becoming the main bottleneck for long-context inference. This is why the industry needs various KV Cache compression techniques (such as quantization, sparsification, and chunk merging) to reduce costs.

Gating layer and slot gain factors: In chunked compression schemes, multiple tokens within a compressed segment are not equally weighted; instead, a learnable gating network assigns a weight to each position. These weights are called slot gain factors, similar to a "soft switch" that determines how much of the K/V information at that position is retained. The model learns on its own during training: certain positions (such as the start of a segment) contain more critical information, so they are assigned high gain; certain positions (such as the middle of a segment) are more redundant, so they are assigned low gain. This data-driven allocation is not perfect, which creates "strong slots" and "blind-spot slots."

Needle-in-a-Haystack Test: A 40% Accuracy Fluctuation

To verify this wasn't a coincidence, the ByteDance team conducted a rigorously controlled "needle-in-a-haystack" experiment.

They constructed a 128K-token context containing about 16,000 key-value pairs. The key-value relationships, the question, and the total context length all remained the same. The only change was: placing one of the key-value pairs at different positions.

The results were striking: DeepSeek-V4-Flash-Base showed a maximum accuracy gap of 40.2% between different positions, and even with DeepSeek-V4-Pro-Base the gap was still 34.8%. Even after post-training optimization, the gaps remained at 19.1% and 14.8%.

What does this mean? If you ask a large model to review dozens of pages of a contract, and a key clause happens to fall in an "information blind spot," the model may simply ignore that information and draw a completely opposite conclusion. You might never know where the problem lies.

Needle-in-a-Haystack is a classic benchmark for evaluating the information retrieval capability of long-context models. The method is: randomly insert a key piece of information (the "needle") into a long stretch of irrelevant text (the "haystack"), then ask the model a question related to that information to test whether it can answer accurately. By varying the absolute position and depth of the needle in the input, one can plot a curve of accuracy versus position. This test can expose differences in a model's retrieval ability at different positions in a long context. In this paper, the ByteDance team further controlled the variables—key-value relationships, question, and total length all unchanged, changing only the position of one key-value pair—thereby isolating the factor of "position" and precisely measuring "phase sensitivity."

Halving the Step Size Is Only a Stopgap

In V4.1-Flash, DeepSeek tried halving the compression step from 4 to 2, merging only 2 tokens per compression round. This way, even if key information falls in a weaker odd position, the adjacent position is only 1 token away, making it hard for the context to be completely lost.

The effect was real—the accuracy gap dropped from 40% to 6.1%. But the problem was not eliminated.

RoPE (Rotary Position Embedding) is a position encoding scheme widely adopted by current mainstream large models. It injects positional information into attention computation through rotation matrices, allowing the model to perceive the relative distance between tokens. A standard Transformer is theoretically translation-invariant, meaning attention depends only on relative position, not absolute position. However, chunked compression breaks this property: the compressed representation depends on the token's absolute position within the compression block (phase), thereby introducing periodic bias. The ByteDance team ruled out interference from position encoding by removing RoPE, proving that the fluctuation stems from the compression mechanism itself.

Not Just a DeepSeek Problem

The ByteDance team trained a batch of models from scratch using the Qwen3-0.6B architecture, with all configurations identical except for the compression mechanism. The conclusions were very clear:

• All models using chunked compression exhibited periodic accuracy fluctuations; the full-attention baseline model without compression was completely normal
• The fluctuation period was strictly equal to the compression step size
• Removing RoPE position encoding, the fluctuation still existed—it was not a bias introduced by position encoding
• Replacing the learnable gating weights with the simplest uniform averaging still produced periodic fluctuations—it was not a matter of poor parameter tuning

As long as chunked compression is done at fixed lengths, this problem is unavoidable. This is a structural flaw of the scheme itself, not something that can be eliminated by tuning parameters, changing position encoding, or post-training.

It is understood that open-source models such as the Llama 3 series, which adopt similar chunked compression or sparsification techniques, more or less suffer from similar issues. In this era of loudly advocating "driving down the cost of long text," this is almost a common blind spot for all models taking this route.

Dynamic Chunking: Antidote or Direction?

Some netizens pointed out that to thoroughly solve this problem, the chunking itself must become "alive"—dynamically determining boundaries based on the importance of the content. When tokens no longer have a fixed phase, the phase difference no longer exists.

The core idea of dynamic chunking is to abandon the one-size-fits-all model of fixed token counts and flexibly adjust the granularity of segmentation based on semantics and importance. Its technical reconstruction involves three levels:

The first level: reconstructing the mathematical foundation of attention. Low-information-density filler is merged into large blocks, while key turning points and high-density information are split finely enough. This mathematically reconstructs the translation invariance broken by fixed chunking and also gives the model a kind of "predictive ability"—read fast where you should read fast, and savor carefully where you should savor carefully.

The second level: forcing the evolution of memory management. The underlying traditional "static contiguous matrix" is no longer applicable and must inevitably evolve toward a "topological sparse matrix," relying on dynamic indexing for addressing. The good news is that the dynamic chunk boundaries between different Transformer layers are highly similar and can be directly reused, greatly amortizing addressing overhead.

The third level: solving the hardware alignment challenge. GPUs naturally prefer regular matrix operations, and uneven chunk sizes would cause acceleration units built specifically for matrix computation to idle extensively. The industry already has feasible solutions such as zero padding.

In terms of practical results, ChunkKV proposed by the Hong Kong University of Science and Technology (Guangzhou) team takes the "semantic continuity" route—using consecutive semantic blocks as the basic compression unit, either retaining the whole block or discarding the whole block, avoiding the traditional method's fragmentation of complete sentence structures. In the NIAH benchmark, which best tests long-text retrieval ability, when the KV cache is limited to 128, ChunkKV based on LLaMA-3 achieves an accuracy of 73.8%, far exceeding the 58.9% of the traditional method SnapKV.

Dr. Han Xiao's team at Jina AI focused on optimizing retrieval scenarios, proposing a delayed chunking scheme that lets the model first read the entire text completely, and only splits it at natural semantic boundaries before outputting the result, preserving complete information to the greatest extent.

In the "sparse computation" direction, Microsoft Research Asia's MInference framework targets million-word-scale scenarios, matching the most suitable sparse computation method to different attention heads, achieving up to 10x acceleration in the prefill stage for million-word long contexts without reducing retrieval accuracy.

The Deeper Problem Revealed by This Bug

In the first half of the long-context race, everyone competed on "who can handle longer," and KV compression techniques played an indispensable role. But the essence of long context has never been length for length's sake—when users truly begin to use long context for real tasks such as contract review, code analysis, and legal retrieval, "usable, easy to use, and not losing information" becomes the new KPI.

The value of this ByteDance paper is not in finding a loophole in DeepSeek, but in exposing the structural cost of an entire technical route: the memory and cost we save for long context may ultimately be paid for by users in the form of "key information silently lost." And what's more troublesome is that such errors do not report errors, are invisible, and are hard to reproduce—you don't even know what to check.

Dynamic chunking is highly anticipated, but it imposes reconstruction requirements on the entire technology stack from algorithm mathematics and memory management to underlying hardware, making large-scale deployment difficult in the short term. During the transition period, users may need to maintain extra vigilance toward the outputs of long-context models—especially when the position of key information in the input is uncertain.

Comparison of dynamic chunking and static chunking: Static chunking mechanically cuts by fixed token counts (such as groups of 4), causing each token's phase to be fixed; once key information falls into a blind-spot slot, the information is systematically discarded. Dynamic chunking adaptively determines split points based on semantic boundaries or importance scores, so tokens no longer have a fixed phase, thereby breaking the periodic weak links. But dynamic chunking also brings new challenges: variable-length chunks complicate memory management, requiring a shift from static contiguous matrices to topological sparse matrices and reliance on dynamic indexing for addressing; at the same time, GPUs excel at regular matrix operations, and irregular chunking reduces hardware utilization. The industry partially alleviates this through techniques such as zero padding and boundary reuse, but there is still a gap before large-scale efficient deployment.

One-line Summary

A ByteDance paper reveals that chunked compression models such as DeepSeek suffer from phase sensitivity, making key information easy to lose.

Paragraph Summary

The ByteDance team found that models using chunked KV Cache compression, such as DeepSeek, exhibit "phase sensitivity": tiny perturbations in the input (such as adding a few equals signs) can cause drastic output fluctuations, because a token's position within the compressed segment (phase) determines how much information is retained. In the 128K needle-in-a-haystack test, the accuracy gap between different positions reached as high as 40.2%, and shortening the compression step only alleviated rather than eliminated the problem. The issue stems from a structural flaw of chunked compression and is unrelated to position encoding or parameter tuning, affecting all models on similar technical routes. Dynamic chunking is seen as a potential antidote, but it faces multiple challenges in algorithms, memory, and hardware alignment.