Nucleus Sampling
A decoding strategy that samples from the smallest set of tokens whose cumulative probability reaches a threshold p. It is used to adjust the degree of divergence in output, tending toward determinism at low temperature and toward diversity at high temperature.
Timeline (last 90 days)
Nucleus sampling and top-k sampling are currently the two most commonly used heuristic strategies in LLM decoding, and their hyperparameters (k or p) are static and cannot adaptively adjust according to context complexity.
The fundamental difference between EOS and nucleus sampling and top-k sampling is that EOS does not control quality by truncating the distribution, but instead dynamically adjusts the temperature through entropy maximization under information-theoretic constraints, with guarantees from online learning theory.
Top-p (nucleus sampling) filters out extremely low-probability outputs by truncating the long tail of the probability distribution.
All Facts (3)
Nucleus sampling and top-k sampling are currently the two most commonly used heuristic strategies in LLM decoding, and their hyperparameters (k or p) are static and cannot adaptively adjust according to context complexity.
50%UnverifiedThe fundamental difference between EOS and nucleus sampling and top-k sampling is that EOS does not control quality by truncating the distribution, but instead dynamically adjusts the temperature through entropy maximization under information-theoretic constraints, with guarantees from online learning theory.
50%UnverifiedTop-p (nucleus sampling) filters out extremely low-probability outputs by truncating the long tail of the probability distribution.
50%