Verified75% confidenceFactExact time
A single sequence of 32K context length for LLaMA-2 70B can require approximately 80 GB of KV cache in FP16.
3
Sources
75%
Confidence
Long-term
Relevance
7/19/2026
First Seen
Sources
Related Claims
Verified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB82% similarVerifiedWhen using FP16 precision, the KV Cache for LLaMA-2-7B occupies approximately 0.5 MB per token.79% similarUnverifiedWhen the context of LLaMA-2 7B is extended to 32K tokens, KV Cache GPU memory usage expands to approximately 16 GB, approaching the total memory of a single consumer-grade GPU.77% similarVerifiedA 35B model requires approximately 70GB of VRAM at FP16 precision, far exceeding the RTX 5090's 32GB capacity, so single-GPU deployment requires quantization.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/560956API
curl https://kongchang.com/api/v1/knowledge/claims/560956MCP
get_claim(id=560956)