TVP Loss Function
A family of loss functions based on expected acceptance length, used to train draft models in speculative decoding. A power parameter interpolates between coverage-oriented and acceptance-oriented objectives in order to increase acceptance length.
Timeline (last 90 days)
TVP is a family of loss functions based on expected acceptance length, whose power parameter can interpolate between coverage-oriented and acceptance-oriented objectives.
By annealing the power parameter from zero to one, TVP achieves the highest or tied-highest acceptance length across three draft model architectures and three target models.
During fine-tuning, TVP improves acceptance length by 9.8% compared with CE.
During fine-tuning, TVP improves acceptance length by 6.2% to 12.9% compared with the publicly available DFlash2 training scheme.
TVP also achieves the optimal acceptance length in the block verification setting.
TVP's power-parameter annealing strategy is simple and effective, and is easy to integrate into existing training pipelines.
All Facts (6)
TVP is a family of loss functions based on expected acceptance length, whose power parameter can interpolate between coverage-oriented and acceptance-oriented objectives.
50%UnverifiedBy annealing the power parameter from zero to one, TVP achieves the highest or tied-highest acceptance length across three draft model architectures and three target models.
50%UnverifiedDuring fine-tuning, TVP improves acceptance length by 9.8% compared with CE.
50%UnverifiedDuring fine-tuning, TVP improves acceptance length by 6.2% to 12.9% compared with the publicly available DFlash2 training scheme.
50%UnverifiedTVP also achieves the optimal acceptance length in the block verification setting.
50%UnverifiedTVP's power-parameter annealing strategy is simple and effective, and is easy to integrate into existing training pipelines.
50%