Rao-Blackwellization
A classical variance reduction technique that replaces an estimator with the conditional expectation of an auxiliary random variable given a sufficient statistic. It reduces variance without introducing bias.
Timeline (last 90 days)
SIRS leverages the conditional distributions of all remaining tokens in the block returned by the model's forward pass to Rao-Blackwellize the increment, reducing the estimation variance without requiring extra queries.
Rao-Blackwellization is a variance reduction technique that replaces the original estimator with the conditional expectation of an auxiliary variable given a sufficient statistic, reducing variance without introducing bias.
All Facts (2)
SIRS leverages the conditional distributions of all remaining tokens in the block returned by the model's forward pass to Rao-Blackwellize the increment, reducing the estimation variance without requiring extra queries.
50%UnverifiedRao-Blackwellization is a variance reduction technique that replaces the original estimator with the conditional expectation of an auxiliary variable given a sufficient statistic, reducing variance without introducing bias.
50%