Domain Classifier
A classifier used to distinguish calibration samples from test samples, whose output probabilities can be used to derive density ratios. Its quality is typically measured by discriminative ability, such as AUC.
Timeline (last 90 days)
A better domain classifier can sometimes instead produce a worse miscoverage rate estimate.
The overall discriminative ability of the domain classifier and the final estimation accuracy are not monotonically related.
In the presence of distribution shift, one need not pursue the most powerful domain classifier, but should instead focus on the weighting scheme in score space.
All Facts (3)
A better domain classifier can sometimes instead produce a worse miscoverage rate estimate.
50%UnverifiedThe overall discriminative ability of the domain classifier and the final estimation accuracy are not monotonically related.
50%UnverifiedIn the presence of distribution shift, one need not pursue the most powerful domain classifier, but should instead focus on the weighting scheme in score space.
50%