Unverified50% confidenceFactExact time
LLaMA-3 70B权重约140GB，推理时每生成一个Token都需要完整遍历一次权重
1
Sources
50%
Confidence
Long-term
Relevance
7/19/2026
First Seen
Sources
Related Claims
UnverifiedLLaMA-2-70B's KV Cache for a single request at a context length of 128K can occupy more than 160 GB of VRAM.77% similarUnverifiedTaking LLaMA-2 70B as an example, when processing a 4096-token context with a batch size of 32, the KV Cache can consume over 80 GB of memory.76% similarUnverifiedFor the LLaMA-2 70B model, when processing a context of 4096 tokens, the KV Cache requires approximately 20 GB of system memory at FP16 precision.72% similarUnverifiedAn M4 Max with 128GB unified memory can directly hold large model weights72% similarUnverifiedMeta's LLaMA 2 70B model still requires approximately 35 GB of VRAM to fully load even with 4-bit quantization.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/559601API
curl https://kongchang.com/api/v1/knowledge/claims/559601MCP
get_claim(id=559601)