Unverified50% confidenceBenchmarkExact time
On the same T4, vLLM 0.27.1 took only 5.8 seconds in total for 8 requests after ready; vLLM's startup overhead was about 165 of 171 billed seconds.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/12/2026
First Seen
Valid until: 10/26/2026
Sources
Related Entities
Related Claims
Unverified一个需要20次LLM调用、每次生成200个Token的Agent任务，在30 TPS下需要约133秒完成，在300 TPS下仅需13秒72% similarUnverifiedThe average time to first token (TTFT) of mainstream cloud-based LLM APIs is typically between 500 milliseconds and 2 seconds.69% similarUnverifiedRunning a 27B model on M1 Max with a 32,000-token prompt requires waiting more than 500 seconds before the first character is output, whereas the M5 Max takes less than one minute (approximately 40 seconds with optimized software).68% similarUnverifiedIn the email classification task, TEF 1 (4B) took 1.273 seconds to run through all samples, while GPT needed 2.568 seconds; in the sponsorship inquiry scoring task, TEF 1 took 1.09 seconds while GPT took about 2.58 seconds.67% similarUnverifiedIn this project, loading the model and artifacts from MLflow takes approximately 40 seconds, while the maximum timeout for API Gateway is 30 seconds, making the first invocation almost certain to fail.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012430API
curl https://kongchang.com/api/v1/knowledge/claims/1012430MCP
get_claim(id=1012430)