[KongchangAI]
Concept

LoopFill

A training-free speculative decoding algorithm that combines multi-token suffix drafts with shallow-loop predictions into a single draft tree, used to accelerate autoregressive decoding in looped Transformers.

Timeline (last 90 days)

Nov 17

LoopFill is a training-free speculative decoding algorithm that combines multi-token suffix drafts with shallow-loop predictions into a single draft tree.

Unverified50%
Nov 17

On seven benchmarks, LoopFill achieves an average speedup of 5.85x over greedy autoregressive decoding on Ouro-2.6B.

Unverified50%
Nov 17

On seven benchmarks, LoopFill achieves an average speedup of 5.13x over greedy autoregressive decoding on Huginn-3.5B.

Unverified50%
Nov 17

LoopFill achieves a 40% speedup over the strongest baseline LoopAhead on Ouro-2.6B.

Unverified50%
Nov 17

LoopFill achieves a 25% speedup over the strongest baseline LoopAhead on Huginn-3.5B.

Unverified50%
Nov 17

LoopFill's shared block computes the highest-scoring draft candidates at different loop depths within a single batch, and full-depth verification preserves the model's autoregressive output distribution.

Unverified50%
Nov 17

LoopFill's training-free nature makes it easy to deploy, while full-depth verification guarantees generation quality.

Unverified50%
Nov 17

LoopFill is mainly applicable to recurrent architectures, and its generality to standard Transformers still needs further verification.

Unverified50%

All Facts (8)

Source Articles