Expired50% confidenceFactExact time
For a typical 70-billion-parameter model, the KV cache occupies approximately 0.5 MB of memory per token, and generating 100,000 tokens requires carrying 50 GB of cache.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
Related Claims
UnverifiedFor a typical 7B model with 32 layers and 32 attention heads, storing the KV Cache for each token requires approximately 0.5 MB of VRAM, so generating 1,000 tokens requires an additional ~500 MB of VRAM81% similarUnverifiedThe weight file of a 7-billion-parameter language model occupies approximately 14 GB of storage, while training data often reaches several TB or even tens of TB.80% similarUnverifiedWhen models with hundreds of billions of parameters process a 128K context window, KV Cache can occupy tens of gigabytes of VRAM.77% similarUnverified大模型推理服务的快照数据量可能达到数十GB甚至上百GB76% similarUnverifiedThe KV Cache of a 70B-parameter model with 128K context support can require hundreds of gigabytes of VRAM at full load, exceeding the 80GB VRAM capacity of a single H100.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/43605API
curl https://kongchang.com/api/v1/knowledge/claims/43605MCP
get_claim(id=43605)