Contrastive Decoding
A decoding strategy that runs a strong model together with a weak model, or the same model under different prompts, and subtracts their output logits to amplify the distinctive tendencies of the strong model relative to the weak model. It is commonly used to improve generation quality or control output style.
Timeline (last 90 days)
Decoding adjustments do not touch model weights or internal representations, but instead modify the token selection strategy during the generation stage; contrastive decoding runs a strong model and a weak model simultaneously and takes the difference in their log probabilities as the score for the next token; DoLa compares the output distributions of different layers within a single model.
The core mechanism of DoubtLess is: let an auxiliary small model read the CoT already generated by the large model under two prompts, concise and verbose, respectively, then multiply the difference between the logit vectors under the two prompts by an intensity coefficient and add it to the logits of the main model's decoding at each step.
The core idea of contrastive decoding is to run a strong model and a weak model simultaneously and subtract their output logits to amplify the strong model's distinctive tendencies relative to the weak model.
All Facts (3)
Decoding adjustments do not touch model weights or internal representations, but instead modify the token selection strategy during the generation stage; contrastive decoding runs a strong model and a weak model simultaneously and takes the difference in their log probabilities as the score for the next token; DoLa compares the output distributions of different layers within a single model.
50%UnverifiedThe core mechanism of DoubtLess is: let an auxiliary small model read the CoT already generated by the large model under two prompts, concise and verbose, respectively, then multiply the difference between the logit vectors under the two prompts by an intensity coefficient and add it to the logits of the main model's decoding at each step.
50%UnverifiedThe core idea of contrastive decoding is to run a strong model and a weak model simultaneously and subtract their output logits to amplify the strong model's distinctive tendencies relative to the weak model.
50%