Unverified50% confidenceFactExact time
LLaMA-2-7B使用FP16精度时每个token的KV Cache约占0.5MB
1
Sources
50%
Confidence
Long-term
Relevance
7/12/2026
First Seen
Sources
vLLM推理框架详解：吞吐优化核心原理与面试攻略
bilibiliAI大模型升升6/9/2026
Related Claims
Verified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB82% similarUnverifiedLLaMA-2 70B单条32K上下文长度序列在FP16下KV缓存可达约80GB79% similarUnverified典型7B模型（32层、32头、128维度）在FP16精度下处理2048 token序列、批大小8时，KV Cache约消耗4GB显存75% similarVerifiedLlama-3 70B模型在FP16精度下仅权重就需要约140GB显存74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/489345API
curl https://kongchang.com/api/v1/knowledge/claims/489345MCP
get_claim(id=489345)