EAGLE3
EAGLE is a family of speculative-decoding draft methods based on feature-layer prediction that improves draft quality by reusing hidden-layer features. EAGLE3 is the latest iteration of this family, further improving the accuracy of draft tokens.
Timeline (last 90 days)
On a single H100 and at similar measured accuracy, QwenDLM-4B + ORL reaches 1,298 tokens/s, DFlash 746 tokens/s, and EAGLE3 405 tokens/s (end-to-end system comparison: ORL runs on dlmrt, speculative decoding runs on vLLM).
v0.30.0 launches a new Model Runner V2, bringing EAGLE3-style draft generation into pipeline parallelism.
All Facts (2)
On a single H100 and at similar measured accuracy, QwenDLM-4B + ORL reaches 1,298 tokens/s, DFlash 746 tokens/s, and EAGLE3 405 tokens/s (end-to-end system comparison: ORL runs on dlmrt, speculative decoding runs on vLLM).
50%Unverifiedv0.30.0 launches a new Model Runner V2, bringing EAGLE3-style draft generation into pipeline parallelism.
50%