LoopFill
A training-free speculative decoding algorithm that combines multi-token suffix drafts with shallow-loop predictions into a single draft tree, used to accelerate autoregressive decoding in looped Transformers.
Timeline (last 90 days)
LoopFill is a training-free speculative decoding algorithm that combines multi-token suffix drafts with shallow-loop predictions into a single draft tree.
On seven benchmarks, LoopFill achieves an average speedup of 5.85x over greedy autoregressive decoding on Ouro-2.6B.
On seven benchmarks, LoopFill achieves an average speedup of 5.13x over greedy autoregressive decoding on Huginn-3.5B.
LoopFill achieves a 40% speedup over the strongest baseline LoopAhead on Ouro-2.6B.
LoopFill achieves a 25% speedup over the strongest baseline LoopAhead on Huginn-3.5B.
LoopFill's shared block computes the highest-scoring draft candidates at different loop depths within a single batch, and full-depth verification preserves the model's autoregressive output distribution.
LoopFill's training-free nature makes it easy to deploy, while full-depth verification guarantees generation quality.
LoopFill is mainly applicable to recurrent architectures, and its generality to standard Transformers still needs further verification.
All Facts (8)
LoopFill is a training-free speculative decoding algorithm that combines multi-token suffix drafts with shallow-loop predictions into a single draft tree.
50%UnverifiedOn seven benchmarks, LoopFill achieves an average speedup of 5.85x over greedy autoregressive decoding on Ouro-2.6B.
50%UnverifiedOn seven benchmarks, LoopFill achieves an average speedup of 5.13x over greedy autoregressive decoding on Huginn-3.5B.
50%UnverifiedLoopFill achieves a 40% speedup over the strongest baseline LoopAhead on Ouro-2.6B.
50%UnverifiedLoopFill achieves a 25% speedup over the strongest baseline LoopAhead on Huginn-3.5B.
50%UnverifiedLoopFill's shared block computes the highest-scoring draft candidates at different loop depths within a single batch, and full-depth verification preserves the model's autoregressive output distribution.
50%UnverifiedLoopFill's training-free nature makes it easy to deploy, while full-depth verification guarantees generation quality.
50%UnverifiedLoopFill is mainly applicable to recurrent architectures, and its generality to standard Transformers still needs further verification.
50%