DLite
A speculative decoding framework for agent workloads. It reuses the target model's contextual hidden states to make history-free local predictions and introduces a lightweight sequential head to recover intra-block causal dependencies, thereby substantially improving acceptance length and inference throughput in agent scenarios.
Timeline (last 90 days)
The DLite framework achieves an acceptance length of 2-6 under agentic workloads.
DLite outperforms the current state-of-the-art method DSpark by up to 70% under agentic workloads.
DLite reuses the contextual hidden states produced by the target model during inference for local speculative prediction, without explicitly maintaining historical context.
DLite introduces a lightweight sequential head to recover causal dependencies within the speculative block at low overhead.
DLite adopts a customized two-stage training strategy, first letting each component adapt to its own task separately and then jointly optimizing overall predictive capability.
DLite was experimentally validated in three types of scenarios: general tasks, long-context tasks, and agentic workloads.
All Facts (6)
The DLite framework achieves an acceptance length of 2-6 under agentic workloads.
50%UnverifiedDLite outperforms the current state-of-the-art method DSpark by up to 70% under agentic workloads.
50%UnverifiedDLite reuses the contextual hidden states produced by the target model during inference for local speculative prediction, without explicitly maintaining historical context.
50%UnverifiedDLite introduces a lightweight sequential head to recover causal dependencies within the speculative block at low overhead.
50%UnverifiedDLite adopts a customized two-stage training strategy, first letting each component adapt to its own task separately and then jointly optimizing overall predictive capability.
50%UnverifiedDLite was experimentally validated in three types of scenarios: general tasks, long-context tasks, and agentic workloads.
50%