Unverified50% confidenceBenchmarkExact time
Running SGLang 0.5.21 on a single T4 in Colab to load the Qwen2.5-0.5B-Instruct fp16 model, in one complete cold-start measurement, 88% of the billed time was spent before the first user request completed.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedThe minimum hardware requirement to run the Origin Prompt Assistant is 16 GB of RAM (32 GB for greater stability), and an Nvidia discrete GPU with 16 GB of VRAM can run a 27B quantized model.63% similarVerifiedWhen using FP16 precision, the KV Cache for LLaMA-2-7B occupies approximately 0.5 MB per token.62% similarUnverifiedThe model completes a full inference pass on the RP2350 in approximately 20 seconds at most, and the results can be displayed on a monitor or transferred via USB.60% similarUnverifiedThe UP host tested MiniCPM-2B on a computer with 4G VRAM and achieved a generation speed of 57 tokens per second, with a first-token latency of only 0.22 seconds.60% similarUnverifiedUsing Q4 quantized weights (a modified version of llama.cpp), the generation speed on the first run was only 1.83 tokens per second.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012424API
curl https://kongchang.com/api/v1/knowledge/claims/1012424MCP
get_claim(id=1012424)