Selective Prediction
A machine learning framework that allows a model to abstain from answering when uncertain. It uses a utility function to balance the reward for correct answers against the cost of incorrect ones, enabling an optimal trade-off between coverage and precision.
Timeline (last 90 days)
Selective Prediction is a formal research direction in machine learning whose core idea is to introduce a rejection option for models, allowing the model to output "I don't know" when confidence falls below a threshold.
An abstention mechanism cannot rely solely on the model's self-assessment; it also requires external calibration methods or the variance of ensemble predictions from multiple models to estimate true uncertainty.
Reliable abstention capability across datasets remains an unsolved problem, as methods that work on one dataset may fail on another.
In the selective prediction framework, the utility function is typically written as U = α·P(correct) - β·P(wrong), where α and β represent the reward weight for correct answers and the penalty weight for wrong answers, respectively.
Confidence-based routing is also referred to in academia and industry as 'selective prediction' or 'classification with a reject option,' with the core being granting the model the right not to answer.
All Facts (5)
Selective Prediction is a formal research direction in machine learning whose core idea is to introduce a rejection option for models, allowing the model to output "I don't know" when confidence falls below a threshold.
50%UnverifiedAn abstention mechanism cannot rely solely on the model's self-assessment; it also requires external calibration methods or the variance of ensemble predictions from multiple models to estimate true uncertainty.
50%UnverifiedReliable abstention capability across datasets remains an unsolved problem, as methods that work on one dataset may fail on another.
50%UnverifiedIn the selective prediction framework, the utility function is typically written as U = α·P(correct) - β·P(wrong), where α and β represent the reward weight for correct answers and the penalty weight for wrong answers, respectively.
50%UnverifiedConfidence-based routing is also referred to in academia and industry as 'selective prediction' or 'classification with a reject option,' with the core being granting the model the right not to answer.
50%