Unverified50% confidenceBenchmarkExact time
In SciResearch-Eval, most models' 95% confidence intervals covered the true value in only 20% to 62% of cases.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/17/2026
First Seen
Valid until: 2/15/2027
Sources
Related Entities
Related Claims
UnverifiedThe confidence reported by the model is severely disconnected from actual accuracy: the 95% confidence tier had an actual accuracy of 0%, while the 90% and 70% tiers both had an actual accuracy of 100%.75% similarUnverifiedFlips are concentrated in the regions where the model's own confidence is low: the lowest confidence tertile has a flip rate of 54%, while the highest tertile has only 8%.72% similarUnverified99.3%通过率(283/285)对应的95%置信区间约为97.5%-99.9%(使用Wilson区间估计)72% similarUnverifiedPaul Christiano said that inside the lab, researchers often debate whether the probability of passing smoothly is 95% or 75%, and few would say 99%.71% similarUnverifiedThe CausalPFN interval for the full-sample average effect contains the true value in only 20% of the datasets, far below the expected 95%.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1178923API
curl https://kongchang.com/api/v1/knowledge/claims/1178923MCP
get_claim(id=1178923)