Unverified50% confidenceBenchmarkExact time
The worst case is 8 users simultaneously sending brand-new 128K prompts, with everyone waiting about 2.5 minutes to get the first token
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/9/2026
First Seen
Valid until: 2/7/2027
Sources
Related Entities
Related Claims
UnverifiedWith 8 users each at 128K context, cached subsequent turns need only 0.2 to 2.5 seconds for the first token74% similarUnverifiedIn the long-context archiving experiment, 8 requests each input approximately 261,000+ tokens and each completed approximately 1,022 outputs; the total throughput of the joint decoding window was 214.85 tokens per second, approximately 26.86 per request, but the window was only approximately 2.28 seconds and prefilling was serial or interleaved.67% similarUnverifiedA P95 latency of 10 seconds means that 1 in every 20 users will experience a frustratingly long wait.66% similarUnverifiedIn a test scenario where eight agents each process 7.5k-token documents and a chat request is issued in the second second, Superfluid answers in 0.36 seconds while llama-server outputs no first token for more than 10 seconds, a first-token latency gap of about 30 times.64% similarUnverifiedIn the provider-outage test, the direct-connection approach sent 89 calls to the dead provider within 60 seconds (30 requests each retried 3 times), whereas going through the proxy sent only 5 calls, a 94% reduction, because the circuit breaker opened after 5 consecutive failures.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1150590API
curl https://kongchang.com/api/v1/knowledge/claims/1150590MCP
get_claim(id=1150590)