Conformal Prediction with Fixed Prediction Set Size
A conformal prediction approach that fixes the size of the prediction set and instead estimates the miscoverage rate. It is suited to settings that require strict control over the size of prediction sets.
Timeline (last 90 days)
Backward conformal prediction (BCP) is a method that fixes the prediction set size and instead estimates the miscoverage rate.
Traditional BCP assumes that the calibration data and the test data are exchangeable, which often does not hold in practical applications.
A recent study extends BCP to the covariate shift setting, preserving its theoretical guarantees through importance weighting.
Directly applying BCP under covariate shift leads to bias in the miscoverage rate estimate.
When the density ratio is known, the miscoverage rate guarantee of BCP can be preserved.
The tight miscoverage rate report of BCP depends only on the relative weights assigned to the omitted calibration samples.
A better domain classifier can sometimes instead produce a worse miscoverage rate estimate.
The proposed method improves the overall miscoverage rate estimation performance of BCP under covariate shift.
All Facts (8)
The proposed method improves the overall miscoverage rate estimation performance of BCP under covariate shift.
50%UnverifiedTraditional BCP assumes that the calibration data and the test data are exchangeable, which often does not hold in practical applications.
50%UnverifiedBackward conformal prediction (BCP) is a method that fixes the prediction set size and instead estimates the miscoverage rate.
50%UnverifiedDirectly applying BCP under covariate shift leads to bias in the miscoverage rate estimate.
50%UnverifiedWhen the density ratio is known, the miscoverage rate guarantee of BCP can be preserved.
50%UnverifiedThe tight miscoverage rate report of BCP depends only on the relative weights assigned to the omitted calibration samples.
50%UnverifiedA better domain classifier can sometimes instead produce a worse miscoverage rate estimate.
50%UnverifiedA recent study extends BCP to the covariate shift setting, preserving its theoretical guarantees through importance weighting.
50%