Self-Speculative Decoding
A variant of speculative decoding that uses different parts of the same model (shallow layers, intermediate layers, or early loop steps) as the draft source, avoiding the memory and deployment costs of an additional draft model.
Timeline (last 90 days)
Self-speculative decoding uses different parts of the same model (such as shallow layers, intermediate layers, or early loop steps) as the draft source, requiring no additional draft model.
The benefit of self-speculative decoding depends on the draft acceptance rate: the more accurate the draft, the more pronounced the end-to-end speedup.
All Facts (2)
Self-speculative decoding uses different parts of the same model (such as shallow layers, intermediate layers, or early loop steps) as the draft source, requiring no additional draft model.
50%UnverifiedThe benefit of self-speculative decoding depends on the draft acceptance rate: the more accurate the draft, the more pronounced the end-to-end speedup.
50%