Unverified50% confidenceFactExact time
The inference speed of mainstream cloud-based large models currently typically ranges from tens to one or two hundred tokens per second.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/12/2026
First Seen
Valid until: 12/11/2026
Sources
Related Entities
Related Claims
UnverifiedUnder this configuration, the 35B model achieves a measured single-user peak speed of approximately 80 tokens/second, with concurrent throughput exceeding 500 tokens/second.73% similarUnverifiedLocal model inference is slow (approximately 1–5 tokens/second) while the cloud can reach 50+ tokens/second; local models have a lower capability ceiling but incur no API costs or privacy concerns.70% similarUnverifiedAirLLM 以推理速度为代价，每生成一个 token 耗时数秒乃至数十秒，远慢于高端 GPU 全量加载的每秒数十 token70% similarUnverifiedA 70B-parameter model (140 GB of weights) has a theoretical maximum generation speed of approximately 7 tokens/second at 1 TB/s bandwidth.70% similarUnverified本地推理的首 token 延迟通常在 100-500ms，而云端 API 在网络良好时需要 500-2000ms 的首 token 延迟69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/913974API
curl https://kongchang.com/api/v1/knowledge/claims/913974MCP
get_claim(id=913974)