Unverified50% confidenceFactExact time
The author reproduced it on a T4 with the same model and testing method: at 8 concurrency, a single request decodes at 122–124 tokens/s, and the whole server produces 701–708 tokens/s, likewise about a 5.7-fold difference.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn a pydantic-ai test, after 10 parallel tool calls (21 new blocks), the next request read 6,827 tokens from cache; after 11 calls (23 blocks), the read count was 0 and 7,534 tokens were written again.68% similarUnverifiedUnder conditions of 2944 input tokens and 1024 output tokens, normal mode reaches 29.9 tokens per second, which improves to 48.4 tokens/s after enabling MTP2.68% similarUnverifiedThe same Gemma 4 E2B model varies from 34.9 to 131.9 tokens/s in single-request generation speed across different AWS inference backends, a significant gap.67% similarUnverifiedIn speculative sampling, when the acceptance rate of candidate tokens is high, the effective number of output tokens per forward pass can reach 2–4, thereby improving the overall generation speed by 1.5x–3x.66% similarUnverifiedAccording to Artificial Analysis testing, Grok 4.6 consumes approximately 38k tokens per task at the MXX High tier, while Grok 4.7 surges to 81k tokens.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1014549API
curl https://kongchang.com/api/v1/knowledge/claims/1014549MCP
get_claim(id=1014549)