Decoding Strategy
A method by which large language models sample the next token one at a time according to a probability distribution during text generation, including greedy decoding, beam search, temperature sampling, top-p, and top-k, which affect the diversity and quality of output.
Timeline (last 90 days)
The design of decoding strategies is moving from experience-driven to theory-driven, and information theory can provide a more solid theoretical foundation and clearer performance bounds.
The design of decoding strategies can make more precise use of the log-linear geometric structure of logits rather than relying on rough distributional assumptions.
Decoding strategy refers to how a model selects output from a probability distribution when generating each token; common strategies include greedy decoding, beam search, temperature sampling, top-k sampling and nucleus sampling, among others.
All Facts (3)
The design of decoding strategies is moving from experience-driven to theory-driven, and information theory can provide a more solid theoretical foundation and clearer performance bounds.
50%UnverifiedDecoding strategy refers to how a model selects output from a probability distribution when generating each token; common strategies include greedy decoding, beam search, temperature sampling, top-k sampling and nucleus sampling, among others.
50%UnverifiedThe design of decoding strategies can make more precise use of the log-linear geometric structure of logits rather than relying on rough distributional assumptions.
50%