Verified80% confidenceFactExact time
LLaMA-2-70B's KV Cache for a single request at a context length of 128K can occupy more than 160 GB of VRAM.
4
Sources
80%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
生产级LLM部署指南：模型选型、GPU配置与推理引擎全解析
Related Claims
VerifiedTaking LLaMA-2 70B as an example, when processing a 4096-token context with a batch size of 32, the KV Cache can consume over 80 GB of memory.82% similarVerifiedLLaMA-2 7B's KV Cache occupies approximately 2 GB with a 4096-token context, and can surge to over 16 GB when extended to a 32K context81% similarVerifiedThe KV Cache of a 70B-parameter model with 128K context support can require hundreds of gigabytes of VRAM at full load, exceeding the 80GB VRAM capacity of a single H100.78% similarUnverifiedFor the LLaMA-2 70B model, when processing a context of 4096 tokens, the KV Cache requires approximately 20 GB of system memory at FP16 precision.78% similarUnverifiedLLaMA-3 70B权重约140GB，推理时每生成一个Token都需要完整遍历一次权重77% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/503793API
curl https://kongchang.com/api/v1/knowledge/claims/503793MCP
get_claim(id=503793)