Temperature Scaling
A post-hoc calibration technique for neural networks that learns a single temperature parameter on a validation set to scale the model's output logits, thereby improving the calibration of probability estimates without requiring model retraining.
Timeline (last 90 days)
Calibration and discrimination are two independent metrics: discrimination measures the model's ability to rank positives ahead of negatives (such as ROC-AUC), while calibration measures whether the model's output probabilities match true frequencies; commonly used calibration measures include Expected Calibration Error (ECE) and reliability diagrams, and correction methods include Platt scaling, isotonic regression and temperature scaling.
The calibration toolchain provided by the system includes three methods: expected calibration error (ECE), Brier score and temperature scaling.
The confidence column shipped by Largha-150M-v1 has an ECE of 0.2519, and the built-in recalibration column (temperature scaling) reduces it to 0.0168.
The escalation gate based on the confidence threshold τ must first be recalibrated with temperature scaling, otherwise the threshold is invalid, because the shipped confidence column has an ECE of 0.2519.
概率校准的常见方法包括温度缩放（temperature scaling）、Platt scaling和等渗回归（isotonic regression），需要在一批带标注的真实数据上拟合校准参数。
Temperature scaling calibrates a model by dividing logit values by a learnable scalar parameter T before the softmax; T greater than 1 softens the probability distribution, T less than 1 sharpens the distribution, and the main model does not need to be retrained.
Engineering mitigations for confidence calibration include post-calibrating logprobs via temperature scaling, introducing an independent validation model for scoring, or measuring result consistency through multiple sampling runs.
Technical methods for improving calibration include post-processing approaches such as temperature scaling and isotonic calibration.
In practice, post-processing methods such as temperature scaling and Platt scaling are commonly used to recalibrate confidence scores.
Technical approaches to addressing poor model calibration include post-processing calibration methods such as temperature scaling and Platt Scaling, as well as uncertainty estimation techniques such as Bayesian neural networks and MC Dropout.
All Facts (12)
Calibration and discrimination are two independent metrics: discrimination measures the model's ability to rank positives ahead of negatives (such as ROC-AUC), while calibration measures whether the model's output probabilities match true frequencies; commonly used calibration measures include Expected Calibration Error (ECE) and reliability diagrams, and correction methods include Platt scaling, isotonic regression and temperature scaling.
50%UnverifiedThe calibration toolchain provided by the system includes three methods: expected calibration error (ECE), Brier score and temperature scaling.
50%UnverifiedThe escalation gate based on the confidence threshold τ must first be recalibrated with temperature scaling, otherwise the threshold is invalid, because the shipped confidence column has an ECE of 0.2519.
50%UnverifiedThe confidence column shipped by Largha-150M-v1 has an ECE of 0.2519, and the built-in recalibration column (temperature scaling) reduces it to 0.0168.
50%Unverified概率校准的常见方法包括温度缩放（temperature scaling）、Platt scaling和等渗回归（isotonic regression），需要在一批带标注的真实数据上拟合校准参数。
50%UnverifiedTemperature scaling calibrates a model by dividing logit values by a learnable scalar parameter T before the softmax; T greater than 1 softens the probability distribution, T less than 1 sharpens the distribution, and the main model does not need to be retrained.
50%UnverifiedEngineering mitigations for confidence calibration include post-calibrating logprobs via temperature scaling, introducing an independent validation model for scoring, or measuring result consistency through multiple sampling runs.
50%UnverifiedTechnical methods for improving calibration include post-processing approaches such as temperature scaling and isotonic calibration.
50%UnverifiedIn practice, post-processing methods such as temperature scaling and Platt scaling are commonly used to recalibrate confidence scores.
50%UnverifiedTechnical approaches to addressing poor model calibration include post-processing calibration methods such as temperature scaling and Platt Scaling, as well as uncertainty estimation techniques such as Bayesian neural networks and MC Dropout.
50%Unverified多数LLM的自报置信度存在过度自信问题，使用前需进行温度缩放等校准处理
50%UnverifiedSoftmax输出的概率值并不等同于真实的贝叶斯后验概率，而是被过度优化以拉开类别间距，导致预测概率系统性偏高
50%