Unverified50% confidenceFactExact time
There are two measures of LLM inference speed in tokens/s: per-request token-by-token generation rate and system-level aggregate throughput, and the two values can differ by an order of magnitude.
1
Sources
50%
Confidence
Long-term
Relevance
10/22/2026
First Seen
Sources
Mercury 3: A Diffusion LLM That Scales 40x Without Slowing Down
producthuntProductHunt Daily10/22/2026
Related Entities
Related Claims
Unverifiedtokens/s指标可细分为首token延迟（TTFT）和生成速率（Generation Throughput）两个子维度78% similarUnverifiedLLM推理服务的限流通常不仅基于请求次数，还会基于输入输出的Token总量进行限速74% similarUnverified推理模式的token消耗通常是普通模式的3-10倍，thinking tokens生成通常增加5-30秒延迟73% similarUnverifiedFor large models, "inference efficiency" usually refers to the number of tokens that can be generated per unit time (throughput) and the first-token latency.73% similarUnverifiedRobustness measures whether two LLMs can still be distinguished after progressively removing the most discriminative tokens.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1085413API
curl https://kongchang.com/api/v1/knowledge/claims/1085413MCP
get_claim(id=1085413)