Conceptsequential head
Sequential Head
A module in DLite with very few parameters that is dedicated to modeling causal dependencies among tokens within a single speculative block in sequence. It generates positions one by one inside the block in an autoregressive or lightweight recurrent manner, ensuring intra-block prediction quality while discarding history.
Timeline (last 90 days)
Oct 19
DLite introduces a lightweight sequential head to recover causal dependencies within the speculative block at low overhead.
Unverified50%