Unverified50% confidenceBenchmarkExact time
Under a configuration using INT4 dynamic quantized weights and FP8 for KVCache, the weights occupy 21.9GB of VRAM, KV occupies 8.5GB, and the total KVCache capacity is 850K tokens.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
Verified以LLaMA-3 70B（80层、64头、每头128维）为例，单个token的KV Cache约占2.6MB（FP16精度），32K上下文的KV Cache总量约达83GB80% similarVerifiedTaking a 70B parameter model with FP16 precision and 32 attention head layers as an example, the KV Cache for a single 200K-length request requires approximately 80–120 GB of VRAM, far exceeding the 80 GB VRAM limit of a single A100 GPU.74% similarUnverifiedFor a typical 7B model (32 layers, 32 heads, 128 dimensions) processing a 2048-token sequence at FP16 precision with a batch size of 8, the KV Cache consumes approximately 4 GB of VRAM.73% similarUnverifiedM3原始权重存储下限：BF16约854GB，8-bit约428GB，4-bit约214GB73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/902432API
curl https://kongchang.com/api/v1/knowledge/claims/902432MCP
get_claim(id=902432)