Eagle-4
The latest iteration in the Eagle series of speculative decoding methods. A lightweight draft model generates candidate tokens, which are then verified in parallel by the target large model, thereby improving the throughput of autoregressive generation.
Timeline (last 90 days)
Eagle-4 has been officially released.
On the Qwen3.8-27B model, Eagle-4 achieves a 3.59x inference speedup relative to Vanilla (original autoregressive decoding).
Eagle-4 currently supports only the Qwen3.8-27B model and does not yet support other mainstream models such as Llama and Mistral.
The poster said they will closely follow the progress of Eagle-4 landing in llama.cpp and vLLM.
For Eagle-4 to land in llama.cpp and vLLM, it needs to implement the construction of the draft tree and parallel verification kernels, and correctly handle the differences in acceptance length across sequences under continuous batching.
The Reddit community reacted positively to Eagle-4's benchmark data, with comments such as "Insane numbers" appearing.
All Facts (6)
On the Qwen3.8-27B model, Eagle-4 achieves a 3.59x inference speedup relative to Vanilla (original autoregressive decoding).
50%UnverifiedEagle-4 currently supports only the Qwen3.8-27B model and does not yet support other mainstream models such as Llama and Mistral.
50%UnverifiedThe poster said they will closely follow the progress of Eagle-4 landing in llama.cpp and vLLM.
50%UnverifiedFor Eagle-4 to land in llama.cpp and vLLM, it needs to implement the construction of the draft tree and parallel verification kernels, and correctly handle the differences in acceptance length across sequences under continuous batching.
50%UnverifiedThe Reddit community reacted positively to Eagle-4's benchmark data, with comments such as "Insane numbers" appearing.
50%UnverifiedEagle-4 has been officially released.
50%