Unverified50% confidenceBenchmarkExact time
The client-side rate limiter allocates send time slots according to the server's replenishment rate, and in the test achieved all 100 succeeding, 100 requests, 0 429s, and the last answer completing at 8.0 seconds.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn the provider-outage test, the direct-connection approach sent 89 calls to the dead provider within 60 seconds (30 requests each retried 3 times), whereas going through the proxy sent only 5 calls, a 94% reduction, because the circuit breaker opened after 5 consecutive failures.79% similarUnverifiedAfter raising llm-retry's max_attempts to 8, all 100 calls passed, but it took 16.6 seconds, more than double the theoretical lower bound, and sent 375 requests.76% similarUnverifiedIn the long-context archiving experiment, 8 requests each input approximately 261,000+ tokens and each completed approximately 1,022 outputs; the total throughput of the joint decoding window was 214.85 tokens per second, approximately 26.86 per request, but the window was only approximately 2.28 seconds and prefilling was serial or interleaved.75% similarUnverifiedIn concurrent scenarios, generation speed is evenly distributed among multiple requests; for example, at 40 token/s for a single request, each of 4 concurrent requests gets approximately 10 token/s74% similarUnverifiedThe retry loop in the code attempts at most 4 times, with exponential backoff starting at 500 milliseconds, and it prioritizes obeying the Retry-After response header when the service provides one.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1064750API
curl https://kongchang.com/api/v1/knowledge/claims/1064750MCP
get_claim(id=1064750)