Verified80% confidenceFactExact time
When using FP16 precision, the KV Cache for LLaMA-2-7B occupies approximately 0.5 MB per token.
4
Sources
80%
Confidence
Long-term
Relevance
7/12/2026
First Seen
Sources
Related Claims
Verified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB82% similarVerifiedA single sequence of 32K context length for LLaMA-2 70B can require approximately 80 GB of KV cache in FP16.79% similarUnverifiedWhen the context of LLaMA-2 7B is extended to 32K tokens, KV Cache GPU memory usage expands to approximately 16 GB, approaching the total memory of a single consumer-grade GPU.76% similarUnverifiedAfter adopting GQA, Llama 3.3 70B has a KV Cache of approximately 320 KB per token.76% similarUnverifiedFor a typical 7B model (32 layers, 32 heads, 128 dimensions) processing a 2048-token sequence at FP16 precision with a batch size of 8, the KV Cache consumes approximately 4 GB of VRAM.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/489345API
curl https://kongchang.com/api/v1/knowledge/claims/489345MCP
get_claim(id=489345)