MTP Speculative Decoding

Multi-Token Prediction speculative decoding is an inference acceleration technique embedded within a model. It predicts multiple subsequent tokens simultaneously and validates them to increase generation speed, yielding approximately 1.6 to 1.8 times faster inference compared to standard autoregressive decoding.