Unverified50% confidenceTradeoffExact time
In concurrent scenarios, generation speed is evenly distributed among multiple requests; for example, at 40 token/s for a single request, each of 4 concurrent requests gets approximately 10 token/s
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/26/2026
First Seen
Valid until: 12/25/2026
Sources
Related Entities
Related Claims
UnverifiedIn the long-context archiving experiment, 8 requests each input approximately 261,000+ tokens and each completed approximately 1,022 outputs; the total throughput of the joint decoding window was 214.85 tokens per second, approximately 26.86 per request, but the window was only approximately 2.28 seconds and prefilling was serial or interleaved.76% similarUnverifiedRate limits are typically measured in requests per minute or per day, while token quotas accumulate based on the combined volume of input and output text.76% similarUnverifiedIt is generally considered that single-concurrency output above 100 tokens/second is sufficient to achieve a smooth typewriter effect, and at approximately 200 tokens/second, generating a 2,000-character long text takes only about 10 seconds.74% similarUnverifiedAfter switching to Max, the average token count surged to 5,000 (more than 10×), average latency rose to 50 seconds, and the slowest response reached 600 seconds.72% similarUnverifiedLocal model inference speed is approximately 1–10 tokens/second, while cloud-based APIs can reach 50–150 tokens/second.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/952596API
curl https://kongchang.com/api/v1/knowledge/claims/952596MCP
get_claim(id=952596)