Unverified50% confidenceFactExact time
KIVI proposes performing per-channel 2-bit quantization on the KV Cache while retaining a small number of outlier channels at full precision, and KVQuant pushes the supported context window to the tens-of-millions-of-tokens scale.
1
Sources
50%
Confidence
Long-term
Relevance
9/26/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOptimization strategies for KV Cache also include KV Cache quantization, latent dimension sharing, and sliding window, which can be used in combination.73% similarVerifiedThe VRAM usage of KV Cache grows linearly with sequence length; for a 70-billion-parameter model using GQA, the KV Cache usage can reach tens of GB when processing 100,000 tokens.71% similarVerifiedWhen models with hundreds of billions of parameters process a 128K context window, KV Cache can occupy tens of gigabytes of VRAM.69% similarUnverifiedWhen the context length is extended from 4K to 8K, the VRAM occupied by the KV Cache nearly doubles68% similarUnverified在长上下文场景（如128K token窗口模型）中，KV Cache的显存占用可能超过模型权重本身68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/952789API
curl https://kongchang.com/api/v1/knowledge/claims/952789MCP
get_claim(id=952789)