Unverified50% confidenceFactExact time
The formula for the number of KV cache bytes that must be moved per token is: kv_bytes_per_token = 2 * layers * kv_heads * head_dim * dtype_bytes, where the factor 2 corresponds to the two copies for Key and Value.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe KV cache calculation formula for Llama 3.3 70B is: 2 (keys and values) × 80 layers × 8 KV heads × 128 head dimension × 2 bytes = 327,680 bytes per token.76% similarUnverifiedFor a 70B model using GQA (n_layers=80, n_kv_heads=8, head_dim=128, FP16 cache), the KV Cache size per token is about 320 KB.73% similarUnverifiedThe formula for estimating the KV cache of a standard full-attention decoder layer is KV bytes = 2 × L × Hkv × D × T × C × S, where 2 stands for keys and values, L is the number of attention layers, Hkv is the number of key/value heads per layer, D is the dimension of each key/value head, T is the number of tokens cached per sequence, C is the number of bytes per cache element, and S is the number of independent sequences.73% similarVerifiedThe size of the KV cache is determined by three multiplied dimensions: the size of each record, how many tokens are combined into one record, and whether layers can share caches with each other.72% similarUnverifiedThe KV cache size can be expressed with the formula cache = 2 (K and V) × layers × KV heads × head dim × tokens × 2 bytes, where 2 bytes corresponds to FP16, doubling when using FP32, and halving when using INT8 quantization.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1069153API
curl https://kongchang.com/api/v1/knowledge/claims/1069153MCP
get_claim(id=1069153)