Unverified50% confidenceFactExact time
The formula for estimating the KV cache of a standard full-attention decoder layer is KV bytes = 2 × L × Hkv × D × T × C × S, where 2 stands for keys and values, L is the number of attention layers, Hkv is the number of key/value heads per layer, D is the dimension of each key/value head, T is the number of tokens cached per sequence, C is the number of bytes per cache element, and S is the number of independent sequences.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe KV cache calculation formula for Llama 3.3 70B is: 2 (keys and values) × 80 layers × 8 KV heads × 128 head dimension × 2 bytes = 327,680 bytes per token.76% similarVerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.75% similarVerifiedThe size of the KV cache is determined by three multiplied dimensions: the size of each record, how many tokens are combined into one record, and whether layers can share caches with each other.72% similarVerifiedKV Cache reduces the computational complexity of self-attention from O(N²) to O(N) by caching already-computed Key-Value pairs in GPU memory, but memory usage grows linearly with sequence length.70% similarUnverifiedThe computational cost of long-context attention grows quadratically with sequence length, and Key and Value vectors must be cached for each token (KV Cache), which at a 1M context can reach hundreds of GB.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1057303API
curl https://kongchang.com/api/v1/knowledge/claims/1057303MCP
get_claim(id=1057303)