Masking
A core implementation strategy for constrained decoding. At each generation step, the probabilities of tokens that violate the constraints are set to negative infinity or zero, after which the remaining tokens are re-normalized and sampled.
Timeline (last 90 days)
Masking is the most common implementation of constrained decoding: setting the probabilities of tokens that do not meet the current constraint to negative infinity or 0, then renormalizing, so that the sampling probability of banned tokens is strictly zero.
The stripping strategy (generate first and then truncate) may corrupt already-generated content, so the mainstream approach is pre-emptive masking rather than post-hoc truncation.
All Facts (2)
Masking is the most common implementation of constrained decoding: setting the probabilities of tokens that do not meet the current constraint to negative infinity or 0, then renormalizing, so that the sampling probability of banned tokens is strictly zero.
50%UnverifiedThe stripping strategy (generate first and then truncate) may corrupt already-generated content, so the mainstream approach is pre-emptive masking rather than post-hoc truncation.
50%