Block Verification
A variant of speculative decoding in which the draft model generates a fixed-length block of tokens at once and the target model verifies all tokens in the block in parallel with rejection sampling, making fuller use of the target model's parallel computation capability.
Timeline (last 90 days)
Block verification is a variant of speculative decoding in which the draft model generates a fixed-length block of tokens at once, and the target model computes probabilities for all tokens in the block in parallel and performs verification and rejection sampling.
TVP also achieves the optimal acceptance length in the block verification setting.
All Facts (2)
Block verification is a variant of speculative decoding in which the draft model generates a fixed-length block of tokens at once, and the target model computes probabilities for all tokens in the block in parallel and performs verification and rejection sampling.
50%UnverifiedTVP also achieves the optimal acceptance length in the block verification setting.
50%