Parallel Block Drafting
A variant of speculative decoding in which the draft model outputs predictions for multiple future positions in a single forward pass rather than generating them autoregressively one at a time, thereby improving drafting efficiency.
Timeline (last 90 days)
HyFlash is a compact parallel draft model for on-device speculative decoding, whose core design is to introduce a prefix memory prior that decouples long-range context modeling from fine-grained draft position interaction.
Parallel block drafting lets the draft model output predictions for multiple positions in a single forward pass rather than generating them one by one autoregressively, thereby improving drafting efficiency.
All Facts (2)
HyFlash is a compact parallel draft model for on-device speculative decoding, whose core design is to introduce a prefix memory prior that decouples long-range context modeling from fine-grained draft position interaction.
50%UnverifiedParallel block drafting lets the draft model output predictions for multiple positions in a single forward pass rather than generating them one by one autoregressively, thereby improving drafting efficiency.
50%