SIRS
A rejection sampling method for masked diffusion models that randomly samples increments and uses Rao-Blackwellization to estimate the log acceptance probability of an entire block within a single model query, reducing query cost and clipping bias.
Timeline (last 90 days)
SIRS (Single-Increment Rejection Sampler) randomly samples one increment and uses a single query to estimate the log acceptance probability of the entire block, thereby reducing query cost while preserving unbiasedness.
SIRS leverages the conditional distributions of all remaining tokens in the block returned by the model's forward pass to Rao-Blackwellize the increment, reducing the estimation variance without requiring extra queries.
SIRS uses the Bernoulli factory proposed by Chen et al. (2026) to convert Poisson-distributed clipped estimates into per-token accept/reject decisions.
SIRS provides an error bound: the KL divergence between the data distribution and the SIRS output distribution is bounded by a clipping term plus the model's training error.
SIRS's sampling error bound shows that the extra bias introduced by the sampling algorithm depends only on the clipping range and does not grow uncontrollably with the block size or the degree of parallelism.
Under exact scores, parallel decoding must pay the full cost of within-block dependencies, whereas SIRS only needs to pay the portion of that dependency cost that exceeds the clipping range.
The effectiveness of SIRS depends on the quality of the score estimates and the choice of the clipping threshold, and how to tune these two in real deployment remains a question that engineering implementation needs to answer.
All Facts (7)
SIRS (Single-Increment Rejection Sampler) randomly samples one increment and uses a single query to estimate the log acceptance probability of the entire block, thereby reducing query cost while preserving unbiasedness.
50%UnverifiedSIRS leverages the conditional distributions of all remaining tokens in the block returned by the model's forward pass to Rao-Blackwellize the increment, reducing the estimation variance without requiring extra queries.
50%UnverifiedSIRS uses the Bernoulli factory proposed by Chen et al. (2026) to convert Poisson-distributed clipped estimates into per-token accept/reject decisions.
50%UnverifiedSIRS provides an error bound: the KL divergence between the data distribution and the SIRS output distribution is bounded by a clipping term plus the model's training error.
50%UnverifiedSIRS's sampling error bound shows that the extra bias introduced by the sampling algorithm depends only on the clipping range and does not grow uncontrollably with the block size or the degree of parallelism.
50%UnverifiedUnder exact scores, parallel decoding must pay the full cost of within-block dependencies, whereas SIRS only needs to pay the portion of that dependency cost that exceeds the clipping range.
50%UnverifiedThe effectiveness of SIRS depends on the quality of the score estimates and the choice of the clipping threshold, and how to tune these two in real deployment remains a question that engineering implementation needs to answer.
50%