Unverified50% confidenceFactExact time
The KV cache calculation formula for Llama 3.3 70B is: 2 (keys and values) × 80 layers × 8 KV heads × 128 head dimension × 2 bytes = 327,680 bytes per token.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTaking the LLaMA-3 70B model as an example, with 80 layers, head_dim=128, and BF16 precision, each token occupies approximately 80KB of KV cache.74% similarVerifiedTaking the LLaMA-2 70B model as an example, a sequence of 2048 tokens requires approximately 4 GB of VRAM for the KV cache alone.73% similarVerifiedThe size of the KV cache is determined by three multiplied dimensions: the size of each record, how many tokens are combined into one record, and whether layers can share caches with each other.72% similarVerifiedThe size of the KV cache grows linearly with context length; the KV cache for Qwen 2.5 32B at a 64K context may consume several additional gigabytes of memory.70% similarVerifiedLLaMA-2 7B's KV Cache occupies approximately 2 GB with a 4096-token context, and can surge to over 16 GB when extended to a 32K context69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1019925API
curl https://kongchang.com/api/v1/knowledge/claims/1019925MCP
get_claim(id=1019925)