Unverified50% confidenceFactExact time
The confidence scores output by language models are softmax-scaled probabilities reflecting relative preference rather than calibrated true probabilities, and models that have undergone RLHF or instruction fine-tuning are generally poorly calibrated.
1
Sources
50%
Confidence
Long-term
Relevance
10/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe confidence output of large language models is fundamentally a product of probability distributions rather than a genuine judgment of factual truth or falsehood, and may express high confidence in incorrect statements.73% similarUnverifiedIn RLHF, if preference data is of low quality or contains systematic bias, the reward model will learn incorrect signals, causing the model's behavior to deviate from expectations.72% similarUnverifiedLarge language models can approximately estimate decision confidence by analyzing token probability distributions, explicitly outputting self-evaluation scores, or observing answer consistency across multiple samplings.72% similarVerifiedThe essence of large language models is learning the conditional probability distribution of what the next word is given the preceding text72% similarUnverifiedTraditional LLMs, after RLHF human preference alignment, tend to give confidently worded answers, which distorts probability estimates.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/973503API
curl https://kongchang.com/api/v1/knowledge/claims/973503MCP
get_claim(id=973503)