Calibration Accuracy
A metric for measuring the quality of a model's decisions under uncertainty. It examines both answer correctness and the degree of confidence calibration, and is used to distinguish blind guessing from reliable evidence-based inference.
Timeline (last 90 days)
Calibration accuracy is a key metric for evaluating an agent's decision quality under uncertainty; it not only examines whether the final answer is correct but also measures whether confidence matches the true probability of correctness.
GPT-6 Sol achieves only 61.9% calibration accuracy on AdaExploreBench.
All Facts (2)
Calibration accuracy is a key metric for evaluating an agent's decision quality under uncertainty; it not only examines whether the final answer is correct but also measures whether confidence matches the true probability of correctness.
50%UnverifiedGPT-6 Sol achieves only 61.9% calibration accuracy on AdaExploreBench.
50%