greedy decoding
A decoding strategy that selects the highest-probability token at each step. The procedure is deterministic in theory, but its outputs are affected by the underlying floating-point computation, so repeated runs may differ depending on hardware and numerical conditions.
Timeline (last 90 days)
Greedy decoding itself is deterministic, but the logits computation it relies on is not deterministic, because changes in batch shape alter the summation order of floating-point operations.
After switching to greedy decoding, the final model frequently fell into loops, a failure mode that was completely invisible in the validation loss
Greedy decoding selects the highest-probability token at each step and is most effective at exposing the model's tendency toward repetition; sampling combined with a temperature parameter re-normalizes logits by dividing by temperature to trade off diversity against coherence
All Facts (3)
Greedy decoding itself is deterministic, but the logits computation it relies on is not deterministic, because changes in batch shape alter the summation order of floating-point operations.
50%UnverifiedAfter switching to greedy decoding, the final model frequently fell into loops, a failure mode that was completely invisible in the validation loss
50%UnverifiedGreedy decoding selects the highest-probability token at each step and is most effective at exposing the model's tendency toward repetition; sampling combined with a temperature parameter re-normalizes logits by dividing by temperature to trade off diversity against coherence
50%