Concept核采样 / Nucleus Sampling / Top-K采样

Top-P Sampling

A sampling strategy used in large language model inference to control output diversity. Top-P (nucleus sampling) draws from the set of candidate tokens whose cumulative probability exceeds a threshold P, while Top-K sampling draws from the K highest-probability candidates. Both methods balance generation quality and diversity.

Timeline (last 90 days)

Sep 10

Sampling logic (Top-K, Top-P, Softmax) is executed on the CPU, and since it only processes a single probability vector (approximately 200 KB of data), the time taken is typically less than 5 ms

Unverified50%

All Facts (1)

Unverified

Sampling logic (Top-K, Top-P, Softmax) is executed on the CPU, and since it only processes a single probability vector (approximately 200 KB of data), the time taken is typically less than 5 ms

50%

Source Articles

Translation Model Stuck in a Degeneration Loop? A Troubleshooting Guide for Rising Loss and Repetitive OutputOct 7 A New Paradigm for AI Testing: The Mindset Shift from Assertions to EvaluationsSep 12