Temperature Scaling
A post-processing model calibration technique that adjusts a model's output probability distribution by dividing the logits by a scalar temperature parameter. This rescaling brings predicted probabilities closer to empirical frequencies without altering the rank ordering of the model's predictions, making it a straightforward method for improving model calibration.
Core Facts
Neural networks have an overconfidence defect: even when inputs fall outside the training distribution, the Softmax layer may still output inflated confidence scores.
Probability calibration refers to a post-processing technique that converts model output prediction scores into true probability estimates, with commonly used methods including Platt Scaling and Isotonic Regression.
Temperature scaling softens the prediction distribution by dividing by a learned temperature parameter T before the softmax, but its applicability to LLMs is limited by the unstructured nature of the output.
Timeline (last 90 days)
Post-processing methods such as Temperature Scaling and Platt Scaling can re-adjust the output probability distribution without changing the model's prediction ranking.
Common post-calibration techniques include Temperature Scaling, Platt Scaling, and Isotonic Regression, which remap raw outputs to more reliable probability estimates by fitting correction functions on an independent validation set without retraining the model.
Confidence calibration methods include post-processing techniques such as Platt Scaling and Temperature Scaling.
Technical methods for improving calibration include post-processing approaches such as temperature scaling and isotonic calibration.
Probability calibration refers to a post-processing technique that converts model output prediction scores into true probability estimates, with commonly used methods including Platt Scaling and Isotonic Regression.
All Facts (7)
Neural networks have an overconfidence defect: even when inputs fall outside the training distribution, the Softmax layer may still output inflated confidence scores.
75%VerifiedProbability calibration refers to a post-processing technique that converts model output prediction scores into true probability estimates, with commonly used methods including Platt Scaling and Isotonic Regression.
65%VerifiedTemperature scaling softens the prediction distribution by dividing by a learned temperature parameter T before the softmax, but its applicability to LLMs is limited by the unstructured nature of the output.
65%UnverifiedCommon post-calibration techniques include Temperature Scaling, Platt Scaling, and Isotonic Regression, which remap raw outputs to more reliable probability estimates by fitting correction functions on an independent validation set without retraining the model.
70%UnverifiedPost-processing methods such as Temperature Scaling and Platt Scaling can re-adjust the output probability distribution without changing the model's prediction ranking.
50%UnverifiedConfidence calibration methods include post-processing techniques such as Platt Scaling and Temperature Scaling.
50%UnverifiedTechnical methods for improving calibration include post-processing approaches such as temperature scaling and isotonic calibration.
50%