Unverified50% confidenceBenchmarkExact time
After adopting GQA, Llama 3.3 70B has a KV Cache of approximately 320 KB per token.
1
Sources
50%
Confidence
Long-term
Relevance
9/23/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedWhen using FP16 precision, the KV Cache for LLaMA-2-7B occupies approximately 0.5 MB per token.76% similarVerified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB75% similarUnverifiedWhen the context of LLaMA-2 7B is extended to 32K tokens, KV Cache GPU memory usage expands to approximately 16 GB, approaching the total memory of a single consumer-grade GPU.72% similarUnverifiedThe KV cache overhead per token for Llama 3.1 8B is approximately 128 KB.70% similarUnverifiedThe PagedAttention paper uses OPT-13B in FP16 as an example, where each token requires approximately 800 KB, and 2048 tokens require approximately 1.6 GB of KV Cache.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/942620API
curl https://kongchang.com/api/v1/knowledge/claims/942620MCP
get_claim(id=942620)