HyFlash
A compact parallel draft model for on-device speculative decoding. It uses a dual-path structure combining local attention and recurrent memory, introduces a prefix-memory prior, and models both long-range dependencies and fine-grained draft-position interactions under limited memory.
Timeline (last 90 days)
HyFlash is a compact parallel draft model for on-device speculative decoding, whose core design is to introduce a prefix memory prior that decouples long-range context modeling from fine-grained draft position interaction.
HyFlash adopts a dual-path design: the local attention path is responsible for capturing interactions between recent context and draft positions; the recurrent memory path compresses historical target features into a fixed-size state.
According to the paper's experimental results, HyFlash surpasses the full-context attention scheme in draft acceptance rate while substantially reducing KV storage requirements.
The paper points out that HyFlash's acceptance rate gains are more pronounced in shallow draft models and long-context scenarios.
All Facts (4)
HyFlash is a compact parallel draft model for on-device speculative decoding, whose core design is to introduce a prefix memory prior that decouples long-range context modeling from fine-grained draft position interaction.
50%UnverifiedHyFlash adopts a dual-path design: the local attention path is responsible for capturing interactions between recent context and draft positions; the recurrent memory path compresses historical target features into a fixed-size state.
50%UnverifiedAccording to the paper's experimental results, HyFlash surpasses the full-context attention scheme in draft acceptance rate while substantially reducing KV storage requirements.
50%UnverifiedThe paper points out that HyFlash's acceptance rate gains are more pronounced in shallow draft models and long-context scenarios.
50%