Unverified50% confidenceBenchmarkExact time
The prefill phase uses small expert-parallel groups of 2-8 GPUs; when the batch increases from about 4,000 tokens to 16,000 tokens, throughput rises from about 60,000 tokens/s to 82,000 tokens/s, and per-step latency increases from about 70 milliseconds to 200 milliseconds.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
Related Entities
Related Claims
UnverifiedUnder the same experimental setup, the total time to generate 1016 tokens drops from 14.66 seconds to 5.03 seconds.77% similarUnverifiedThe time required for a million-token output may reach the level of several minutes, and integration requires designing asynchronous calls and progress feedback mechanisms.74% similarUnverifiedIn a case test, Gemma took approximately 13.78 seconds to analyze a three-page PDF financial report, generating 1,064 tokens at a speed of 16 tokens per second.74% similarUnverifiedIn this test, the prefill speed was fixed at approximately 500–600 tokens/s, and the time to first token increased linearly with input length.73% similarUnverifiedOn an 8GB GPU, the token generation speed is approximately 4.2 to 5 tokens/second, and decreases from around 6 tokens/second to approximately 4.2 as the context grows longer.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1136865API
curl https://kongchang.com/api/v1/knowledge/claims/1136865MCP
get_claim(id=1136865)