[KongchangAI]
Conceptself-speculative decoding

Self-Speculative Decoding

A variant of speculative decoding that uses different parts of the same model (shallow layers, intermediate layers, or early loop steps) as the draft source, avoiding the memory and deployment costs of an additional draft model.

Timeline (last 90 days)

Nov 19

Self-speculative decoding uses different parts of the same model (such as shallow layers, intermediate layers, or early loop steps) as the draft source, requiring no additional draft model.

Unverified50%
Nov 19

The benefit of self-speculative decoding depends on the draft acceptance rate: the more accurate the draft, the more pronounced the end-to-end speedup.

Unverified50%

All Facts (2)

Source Articles