Medusa
A speculative decoding variant that adds multiple parallel decoding heads on top of the target model's final layer, representing one implementation approach to speculative decoding.
Core Facts
Timeline (last 90 days)
Medusa achieves self-drafting by attaching multiple decoding heads to the target model.
Speculative decoding has multiple variants, including Medusa, EAGLE/EAGLE-2, Lookahead Decoding, and SpecInfer, each with different trade-offs in acceptance rate, complexity, VRAM overhead, and architectural invasiveness.
Speculative decoding has spawned multiple variants including Medusa (multi-head prediction), EAGLE (generating drafts using hidden states from early layers of the main model), and DFlash (hybrid layout optimization).
Speculative decoding variants such as Medusa and Eagle further reduce the dependency on a separate draft model.
Medusa adds multiple prediction heads on top of a model to predict tokens at different future positions and confirms them through tree-based verification, without requiring an additional draft model, and can improve inference speed by 2–3×.
Speculative decoding generates drafts with a small model and verifies them in batch with a large model; mathematically it guarantees output fully consistent with the large model through rejection sampling, making it lossless acceleration
All Facts (6)
Medusa adds multiple prediction heads on top of a model to predict tokens at different future positions and confirms them through tree-based verification, without requiring an additional draft model, and can improve inference speed by 2–3×.
75%UnverifiedSpeculative decoding has multiple variants, including Medusa, EAGLE/EAGLE-2, Lookahead Decoding, and SpecInfer, each with different trade-offs in acceptance rate, complexity, VRAM overhead, and architectural invasiveness.
70%UnverifiedSpeculative decoding generates drafts with a small model and verifies them in batch with a large model; mathematically it guarantees output fully consistent with the large model through rejection sampling, making it lossless acceleration
70%UnverifiedMedusa achieves self-drafting by attaching multiple decoding heads to the target model.
50%UnverifiedSpeculative decoding has spawned multiple variants including Medusa (multi-head prediction), EAGLE (generating drafts using hidden states from early layers of the main model), and DFlash (hybrid layout optimization).
50%UnverifiedSpeculative decoding variants such as Medusa and Eagle further reduce the dependency on a separate draft model.
50%