tok/s
The core unit for measuring decoding speed in large-model inference, denoting the number of tokens generated per second; it directly affects conversational fluency.
Timeline (last 90 days)
On agent turns, with 1 concurrent agent the median is 186 tok/s and p5 is 134 tok/s; with 4 it is 59 tok/s median and 52 tok/s p5; with 6 it is 40 tok/s median and 34 tok/s p5. The article sets 40 tok/s per person as the standard, and four concurrent streams with 10% headroom is this machine's capacity.
20 tok/s左右接近跟得上阅读的体验，6 tok/s则明显有逐字蹦出的迟滞感。
A single throughput figure (tok/s) cannot reflect a model's true capability, long-context cost, or cross-model comparability.
在流式对话场景中，模型生成速度低于2 tok/s基本无法实现流畅交互
All Facts (4)
On agent turns, with 1 concurrent agent the median is 186 tok/s and p5 is 134 tok/s; with 4 it is 59 tok/s median and 52 tok/s p5; with 6 it is 40 tok/s median and 34 tok/s p5. The article sets 40 tok/s per person as the standard, and four concurrent streams with 10% headroom is this machine's capacity.
50%Unverified20 tok/s左右接近跟得上阅读的体验，6 tok/s则明显有逐字蹦出的迟滞感。
50%UnverifiedA single throughput figure (tok/s) cannot reflect a model's true capability, long-context cost, or cross-model comparability.
50%Unverified在流式对话场景中，模型生成速度低于2 tok/s基本无法实现流畅交互
50%