Unverified50% confidenceSolutionExact time
V4.1 adopts SWA bounded replay: it recomputes only the last 128 tokens, and attention is also confined to within these 128 tokens, yielding an approximate window cache; the recomputation cost becomes constant, and the Decoder's 20 layers, which originally had to process the last 2,560 tokens, now only need to process 128, reducing total prefill computation by nearly half.
1
Sources
50%
Confidence
Long-term
Relevance
10/23/2026
First Seen
Sources
DeepSeek Open-Sources Inference Engine DeepInfer: All the Cost-Saving Tricks Laid Bare
bilibili最佳拍档大飞10/23/2026
Related Entities
Related Claims
VerifiedTaking the LLaMA-2 70B model as an example, a sequence of 2048 tokens requires approximately 4 GB of VRAM for the KV cache alone.69% similarUnverifiedIn an AWS sample application, across 28 calls reviewing the same document, CloudWatch recorded 196,680 tokens written to cache and 0 reads, because the cache point was placed after each entry's own instructions; after moving the cache point to after the document, the second entry immediately became a read of 4,624 tokens.68% similarUnverifiedFor this architecture, the cached bytes per token is not a constant; the equivalent bytes/token of the window layers alone is about 688.0 at a context of 8,192, about 43.0 at 131,072, and about 5.4 at 1,048,576.68% similarUnverifiedThe model natively supports a 256K token context and can be scaled to approximately 1 million tokens via 4x scaling using YaRN technology.66% similarVerifiedIn RAG practice, chunk_size is typically set between 256 and 1024 tokens, with an Overlap of approximately 10% to 20% retained.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1090444API
curl https://kongchang.com/api/v1/knowledge/claims/1090444MCP
get_claim(id=1090444)