Calibrated Decision
A concept from probability theory and decision theory referring to the degree to which a model's output probabilities align with actual observed frequencies—for example, events assigned a 70% probability occurring approximately 70% of the time. Calibration is the core optimization objective in training judgment models.
Core Facts
Timeline (last 90 days)
Existing LLMs optimize for "human preference," whereas new models like Jev optimize for "calibrated decisions"—that is, cognitively honest probabilistic answers.
A "well-calibrated" probability judgment means that when a model says something has a 70% chance of occurring, the actual occurrence rate of that event across many similar predictions is close to 70%.
All Facts (2)
A "well-calibrated" probability judgment means that when a model says something has a 70% chance of occurring, the actual occurrence rate of that event across many similar predictions is close to 70%.
65%UnverifiedExisting LLMs optimize for "human preference," whereas new models like Jev optimize for "calibrated decisions"—that is, cognitively honest probabilistic answers.
50%