Acceptance Rate
In speculative decoding, the proportion of tokens generated by the draft model that are accepted by the target model, a key indicator of draft model quality. The aggregate acceptance rate is computed over all decision points combined and may mask local performance fluctuations.
Core Facts
Timeline (last 90 days)
Acceptance length and acceptance rate do not have a simple linear relationship: it depends on the product of the probabilities that prefix tokens are accepted one by one.
Distilled models further improve alignment by having the draft model fit the target model's output distribution, thereby raising the acceptance rate.
For tasks with strong output determinism (such as code completion and structured generation), the acceptance rate of speculative decoding is usually high; for open-ended creation, the acceptance rate may decline.
A higher acceptance rate means the draft model's guesses are more accurate and the large model needs to redo less work, which directly determines the actual speedup achieved by speculative decoding.
All Facts (4)
A higher acceptance rate means the draft model's guesses are more accurate and the large model needs to redo less work, which directly determines the actual speedup achieved by speculative decoding.
85%UnverifiedAcceptance length and acceptance rate do not have a simple linear relationship: it depends on the product of the probabilities that prefix tokens are accepted one by one.
50%UnverifiedDistilled models further improve alignment by having the draft model fit the target model's output distribution, thereby raising the acceptance rate.
50%UnverifiedFor tasks with strong output determinism (such as code completion and structured generation), the acceptance rate of speculative decoding is usually high; for open-ended creation, the acceptance rate may decline.
50%