Importance Sampling
A sampling technique in statistics based on measure transformation. By introducing an auxiliary proposal distribution that samples more frequently in the target rare region, and reweighting estimates using likelihood ratios, it achieves unbiased estimation of low-probability events.
Timeline (last 90 days)
The importance sampling optimization idea can compress the actual algorithmic complexity from O(N) to O(k), where k is much smaller than N.
Importance sampling (IS) from a normalizing flow is exactly the limiting case of this control variate construction, where the control variate method converges to the importance sampling form when the latent-space sampler degenerates to i.i.d. sampling.
Importance sampling (IS) draws samples from an easy-to-sample proposal distribution q and uses weights w(x) = π(x)/q(x) to correct the bias for estimating the expectation under the target distribution π.
In PPO and GRPO algorithms, the importance sampling correction weight is proportional to the ratio of new and old policy probabilities exp(logprob_new - logprob_old); due to the amplification of the exponential function, a logprob discrepancy of 0.001 can produce a substantial weight error.
RAPID employs an inverse proposal correction technique to ensure the loss function and gradient estimator are unbiased under conditional expectation
All Facts (5)
The importance sampling optimization idea can compress the actual algorithmic complexity from O(N) to O(k), where k is much smaller than N.
50%UnverifiedImportance sampling (IS) from a normalizing flow is exactly the limiting case of this control variate construction, where the control variate method converges to the importance sampling form when the latent-space sampler degenerates to i.i.d. sampling.
50%UnverifiedImportance sampling (IS) draws samples from an easy-to-sample proposal distribution q and uses weights w(x) = π(x)/q(x) to correct the bias for estimating the expectation under the target distribution π.
50%UnverifiedIn PPO and GRPO algorithms, the importance sampling correction weight is proportional to the ratio of new and old policy probabilities exp(logprob_new - logprob_old); due to the amplification of the exponential function, a logprob discrepancy of 0.001 can produce a substantial weight error.
50%UnverifiedRAPID employs an inverse proposal correction technique to ensure the loss function and gradient estimator are unbiased under conditional expectation
50%