ReaDy
A speculative decoding method that uses the recurrent state of hybrid linear attention models for position-specific context retrieval. It retrieves supplementary context for each draft position through a state reader to improve draft quality.
Timeline (last 90 days)
ReaDy is the first speculative decoding method that uses hybrid linear attention (HLA) recurrent states for position-specific context retrieval.
The core idea of ReaDy is to let each draft position retrieve supplementary context from the target model's already-committed recurrent state through a learnable query.
ReaDy adopts low-rank state reading and fuses the relevant operators to reduce extra computation and I/O overhead.
ReaDy is built on top of DSpark's parallel backbone and sequence prediction head, using lightweight two-stage adaptation.
On seven benchmarks covering mathematics, code generation and dialogue, ReaDy using Qwen3.8-27B achieved an average speedup of 3.16x over autoregressive decoding on a single NVIDIA B200 GPU.
Compared with the DSpark baseline, ReaDy improves on all benchmarks, with throughput increasing by an average of 7.97% and accepted length increasing by an average of 8.50%.
The model scales and hardware conditions on which ReaDy has currently been validated are relatively concentrated, and its generalization ability across architectures and scales still requires more experimental verification.
All Facts (7)
ReaDy is the first speculative decoding method that uses hybrid linear attention (HLA) recurrent states for position-specific context retrieval.
50%UnverifiedThe core idea of ReaDy is to let each draft position retrieve supplementary context from the target model's already-committed recurrent state through a learnable query.
50%UnverifiedReaDy adopts low-rank state reading and fuses the relevant operators to reduce extra computation and I/O overhead.
50%UnverifiedReaDy is built on top of DSpark's parallel backbone and sequence prediction head, using lightweight two-stage adaptation.
50%UnverifiedOn seven benchmarks covering mathematics, code generation and dialogue, ReaDy using Qwen3.8-27B achieved an average speedup of 3.16x over autoregressive decoding on a single NVIDIA B200 GPU.
50%UnverifiedCompared with the DSpark baseline, ReaDy improves on all benchmarks, with throughput increasing by an average of 7.97% and accepted length increasing by an average of 8.50%.
50%UnverifiedThe model scales and hardware conditions on which ReaDy has currently been validated are relatively concentrated, and its generalization ability across architectures and scales still requires more experimental verification.
50%