Unverified50% confidenceBenchmarkExact time
以70B参数模型、FP16精度、32层注意力头为例，单条200K长度请求的KV Cache约需80-120GB显存，远超单张A100 GPU的80GB显存上限
1
Sources
50%
Confidence
Long-term
Relevance
7/16/2026
First Seen
Sources
Related Claims
Verified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB79% similarVerified参数量达1750亿的GPT-4级模型，即便以fp8精度存储也需要约175GB显存78% similarUnverified单卡A100（80GB显存）可运行7B-13B级别模型，4卡A100（320GB显存）可运行70B级别模型77% similarVerified一个70B参数的FP16模型大约需要140GB显存，INT8量化后约需70GB，INT4量化后约需35-40GB77% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/526414API
curl https://kongchang.com/api/v1/knowledge/claims/526414MCP
get_claim(id=526414)