Unverified50% confidenceFactExact time
Speculative decoding optimizes inter-token latency (ITL) in the decoding stage and offers essentially no improvement to time to first token (TTFT), which may even increase slightly due to loading the drafter.
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedSpeculative decoding mainly improves throughput and cost per token, with limited improvement to first-token latency and possibly even a slight increase due to the draft stage.79% similarUnverifiedDeterministic execution architecture minimizes variance in time-to-first-token (TTFT) and inter-token latency (ITL), enabling SLA-like latency commitments to be offered externally.77% similarUnverifiedTokenSpeed's MLA decode kernel has about half the latency of TensorRT-LLM under typical speculative decoding workloads.71% similarUnverifiedtokens/s指标可细分为首token延迟（TTFT）和生成速率（Generation Throughput）两个子维度69% similarUnverifiedIn the inference pipeline, the Prefill stage is compute-intensive and time-consuming, while the Decode stage performs token-by-token autoregressive generation and is sensitive to latency.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1106560API
curl https://kongchang.com/api/v1/knowledge/claims/1106560MCP
get_claim(id=1106560)