Unverified50% confidencePredictionExact time
For the Qwen2.5-1.5B model, the int4 theoretical roofline ceiling for batch-1 decoding is 411.8 tok/s, while the actual self-developed kernel reaches only 25.5 tok/s, a gap of about 16x.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedIn the same test, the bf16 baseline exceeded the self-developed W4 kernel (25.5 tok/s) and bitsandbytes NF4 (25.7 tok/s) at 31.3 tok/s, reaching 29.8% of the theoretical roofline.73% similarUnverifiedIn a speculative decoding test using Qwen2.5 0.5B as the draft model proposing 4 tokens at a time, the overall draft acceptance rate was 32.9% for serial processing and 37.1% for concurrent processing.71% similarUnverifiedGPUs with 24–36 GB of VRAM can run quantized versions of larger-parameter models in the Qwen3 series, which is the tier closest to top-tier closed-source models.70% similarUnverifiedThe 4-bit quantization of Qwen3 27B shows robust performance, with the gap from FP16 baseline on mainstream benchmark tasks remaining within an acceptable range.70% similarUnverifiedQwen2.5-Max-0902 has a maximum output of 130,000 tokens, with the limit rising to 260,000 tokens in reasoning mode.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1006835API
curl https://kongchang.com/api/v1/knowledge/claims/1006835MCP
get_claim(id=1006835)