Unverified50% confidenceBenchmarkExact time
When the context of LLaMA-2 7B is extended to 32K tokens, KV Cache GPU memory usage expands to approximately 16 GB, approaching the total memory of a single consumer-grade GPU.
1
Sources
50%
Confidence
Long-term
Relevance
9/15/2026
First Seen
Sources
Related Entities
Related Claims
Verified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB80% similarUnverifiedA single sequence of 32K context length for LLaMA-2 70B can require approximately 80 GB of KV cache in FP16.77% similarVerifiedWhen using FP16 precision, the KV Cache for LLaMA-2-7B occupies approximately 0.5 MB per token.76% similarUnverifiedUnder a configuration using INT4 dynamic quantized weights and FP8 for KVCache, the weights occupy 21.9GB of VRAM, KV occupies 8.5GB, and the total KVCache capacity is 850K tokens.73% similarUnverifiedFor a typical 7B model (32 layers, 32 heads, 128 dimensions) processing a 2048-token sequence at FP16 precision with a batch size of 8, the KV Cache consumes approximately 4 GB of VRAM.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/923484API
curl https://kongchang.com/api/v1/knowledge/claims/923484MCP
get_claim(id=923484)