Unverified50% confidenceBenchmarkExact time
In this test, when reading a 2048-token prompt, FP8 prefill speed was 10,118 tok/s, ahead of BF16 (7,003 tok/s), NVFP4 (6,978 tok/s), MXFP8 (6,061 tok/s) and GGUF Q5_K_M (1,710 tok/s).
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/18/2026
First Seen
Valid until: 1/16/2027
Sources
Related Entities
Related Claims
UnverifiedIn the GSM8K test (1319 questions, 8192-token limit), BF16 scored 1249, FP8 scored 1241, MXFP8 scored 1250, and NVFP4 scored 1238, with the three quantized formats matching BF16 within a ±3 point tolerance.73% similarUnverifiedWith a 32K prompt input, the Q2 quantization read speed can increase from 572 tokens/s to 1,290 tokens/s, and Q3 can increase from 383 to 1,208 tokens/s.69% similarUnverifiedIn Strata's speed table, under a 128k context, Q2_0 (2-bit) achieves 65.1 tokens/s and IQ2_XS achieves 52 tokens/s; under a 4k short context, IQ3_XXS achieves 65.6 tokens/s.69% similarUnverifiedNumerical correctness validation of the engine shows that, compared with the CPU fp32 transformers reference implementation, the English paragraph PPL is 5.233 vs 5.280, average KL ≤ 0.003, and greedy answer matches are 63/64 and 64/64.69% similarUnverifiedIn the same test, the bf16 baseline exceeded the self-developed W4 kernel (25.5 tok/s) and bitsandbytes NF4 (25.7 tok/s) at 31.3 tok/s, reaching 29.8% of the theoretical roofline.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1057657API
curl https://kongchang.com/api/v1/knowledge/claims/1057657MCP
get_claim(id=1057657)