Unverified50% confidenceBenchmarkExact time
Inference optimization reaches up to 4,856 output tokens/s per GPU on GB200 while maintaining an experience floor of at least 50 tokens/s per user.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/30/2026
First Seen
Valid until: 11/13/2026
Sources
Related Entities
Related Claims
UnverifiedSGLang using official NVFP4 weights with D-Spark speculative decoding achieved an average of approximately 173 tokens/second on Blackwell GPUs, approaching 200 for short tasks, while loading a 262k context window.76% similarUnverified缺少独立 GPU 的设备运行 2B 参数模型时生成速度可能低于每秒 5 个 token74% similarUnverifiedUnder this configuration, the 35B model achieves a measured single-user peak speed of approximately 80 tokens/second, with concurrent throughput exceeding 500 tokens/second.73% similarUnverifiedThe core bottleneck in large model inference on GPUs is memory bandwidth; autoregressive generation of each token requires loading the full KV Cache from HBM, with compute utilization typically below 5%.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114697API
curl https://kongchang.com/api/v1/knowledge/claims/1114697MCP
get_claim(id=1114697)