Unverified50% confidenceBenchmarkExact time
Mainstream GPU inference solutions based on the NVIDIA H100 can typically only achieve tens to one or two hundred tokens per second on models of equivalent scale.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/6/2026
First Seen
Valid until: 12/5/2026
Sources
Related Entities
Related Claims
UnverifiedThe GPU resources required for a single inference of a hundred-billion-parameter flagship model may be 10 to 20 times higher than those of a ten-billion-parameter model.74% similarUnverifiedModels with larger parameter counts and more reasoning steps may consume 10–50 times the GPU compute of lightweight models.73% similarUnverifiednvidia-smi报告的GPU利用率本质上只统计采样时间窗口内GPU上至少有一个内核在执行的时间占比，而非计算资源的实际占用率72% similarUnverified一个15亿参数的模型使用Adam优化器时每个GPU至少需要约24GB的状态信息72% similarUnverified相较于用GPU执行AI推理，NPU的功耗通常只有前者的1/5到1/1072% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/863320API
curl https://kongchang.com/api/v1/knowledge/claims/863320MCP
get_claim(id=863320)