Unverified50% confidenceBenchmarkExact time
In the same test, the bf16 baseline exceeded the self-developed W4 kernel (25.5 tok/s) and bitsandbytes NF4 (25.7 tok/s) at 31.3 tok/s, reaching 29.8% of the theoretical roofline.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/12/2026
First Seen
Valid until: 10/26/2026
Sources
Related Entities
Related Claims
UnverifiedFor the Qwen2.5-1.5B model, the int4 theoretical roofline ceiling for batch-1 decoding is 411.8 tok/s, while the actual self-developed kernel reaches only 25.5 tok/s, a gap of about 16x.73% similarUnverifiedThe self-developed W4 kernel's perplexity (probe ppl) is 15.76, better than bitsandbytes NF4's 18.98.73% similarUnverifiedBF16采用1位符号位、8位指数位和7位尾数位，牺牲精度但保留了与FP32相同的动态范围69% similarUnverifiedThe performance gap between a carefully quantized 4-bit model and the original FP16 model is typically within 1–3 percentage points.69% similarUnverified在27B混合架构、4B稠密模型、36B MoE三类模型上以8k上下文测试，Prefill阶段性能无变化，Decode阶段打补丁后快约1.4%68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1006834API
curl https://kongchang.com/api/v1/knowledge/claims/1006834MCP
get_claim(id=1006834)