[KongchangAI]
Concept校准

calibration

A metric measuring the agreement between a model's predicted probabilities and observed frequencies. If a model assigns a 70% risk to a group of patients, approximately 70% of them should actually experience the outcome for the model to be well calibrated.

Core Facts

Timeline (last 90 days)

Nov 4

Calibration is a key metric for evaluating the quality of probabilistic forecasts, referring to the degree of match between predicted probabilities and actual frequencies of occurrence.

Unverified50%
Nov 4

The calibration of market prices is usually repeatedly corrected by a large number of traders, whereas the probability outputs of models are prone to overconfidence or conservatism.

Unverified50%
Oct 10

Most general-purpose large models suffer from overconfidence issues, where their output probabilities reflect token distribution preferences more than genuine uncertainty estimates.

Unverified50%
Sep 22

The calibration of safety guardrails refers to whether the threshold at which a model refuses requests is set appropriately—being overly conservative leads to false rejections of legitimate requests, while insufficient calibration allows harmful content to pass through.

Unverified50%
Sep 12

Model calibration is defined in statistics as: if a model claims a certain answer has 70% confidence, then the actual proportion of correct answers among a large number of such claims should be close to 70%.

Verified85%
Aug 27

研究表明校准能力与模型规模之间存在正相关,这一关系在多项实证研究中得到验证

Unverified50%
Aug 26

There is a positive correlation between a model's calibration capability and model scale; larger models can more finely distinguish between states of knowledge.

Verified75%

All Facts (7)

Source Articles