Unverified50% confidenceFactExact time
H100的内存带宽约3.35TB/s，GPT-4规模模型单次生成一个Token需读取约1TB参数数据，理论上每秒最多生成约3个Token
1
Sources
50%
Confidence
Long-term
Relevance
7/19/2026
First Seen
Sources
Related Claims
UnverifiedGPT-4级别模型的128K上下文窗口的KV缓存可能占用数十GB显存79% similarUnverified在45nm工艺下，一次32位浮点乘法运算消耗约3.7pJ能量，从片外DRAM读取32位数据消耗约640pJ，片上SRAM访问需要约5pJ75% similarUnverified若量化后模型权重为150GB，以273GB/s带宽读取一遍权重约需0.55秒，理论上单token生成速度约为1.8 token/s（未计入计算延迟和跨机通信开销）73% similarUnverified以LLaMA-2 70B模型为例，2048 token的序列需约4GB显存仅用于KV缓存71% similarUnverified百万Token级别的单次推理需要数十至数百GB的高端GPU显存（A100/H100级别）70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/563265API
curl https://kongchang.com/api/v1/knowledge/claims/563265MCP
get_claim(id=563265)