Unverified50% confidenceFactExact time
If this 70B model used classic multi-head attention (MHA, 64 KV heads), the KV Cache under the same conditions would expand to 312.5 GiB.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWith a 64K context, batch=2, and FP16 KV Cache, the KV Cache usage of this 70B model is about 39.06 GiB.75% similarVerifiedFor a model with 70B parameters, the KV Cache for an 8K context may occupy several additional gigabytes of VRAM70% similarVerifiedThe VRAM consumption of the KV cache is proportional to context length, number of model layers, and number of attention heads; under an 8K context, a typical 27B model may consume an additional 2–4 GB of VRAM.70% similarVerifiedTaking LLaMA-2 70B as an example, when processing a 4096-token context with a batch size of 32, the KV Cache can consume over 80 GB of memory.69% similarUnverifiedTaking the LLaMA-3 70B model as an example, with 80 layers, head_dim=128, and BF16 precision, each token occupies approximately 80KB of KV cache.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1025446API
curl https://kongchang.com/api/v1/knowledge/claims/1025446MCP
get_claim(id=1025446)