Unverified50% confidenceFactExact time
Under the batch-size-1 Spark configuration, the cost of a step verifying 4 tokens is roughly equal to 2.26 ordinary decoding steps.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/29/2026
First Seen
Valid until: 1/27/2027
Sources
Related Entities
Related Claims
UnverifiedIn code scenarios, each verification step accepts about 7.4 tokens, reaching a throughput of 154 tok/s; in prose scenarios each step accepts about 2.5 tokens, with a throughput of 35-37 tok/s; the ordinary decoding baseline throughput is about 18-19 tok/s.63% similarUnverifiedFor the Qwen2.5-1.5B model, the int4 theoretical roofline ceiling for batch-1 decoding is 411.8 tok/s, while the actual self-developed kernel reaches only 25.5 tok/s, a gap of about 16x.61% similarUnverifiedInference energy consumption across different model and decoding configuration combinations spans from 0.79 MJ to 8.68 MJ, roughly equivalent to the energy of 250 to 2800 GPT-4.1 queries.61% similarUnverifiedIn a speculative decoding test using Qwen2.5 0.5B as the draft model proposing 4 tokens at a time, the overall draft acceptance rate was 32.9% for serial processing and 37.1% for concurrent processing.61% similarUnverifiedGPT-4o-mini scores about 18% accuracy on the same HS code benchmark; the best AI Agent with web search reaches 46.8%, but costs $1-10 per call and takes 10-30 seconds; human experts are about 95%.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1110402API
curl https://kongchang.com/api/v1/knowledge/claims/1110402MCP
get_claim(id=1110402)