Unverified50% confidenceFactExact time
Calibration concerns whether the probability values are trustworthy, while discriminative power concerns whether high-confidence samples are more likely to be correct than low-confidence samples; even if calibration is biased, as long as confidence maintains ranking discriminative power, it can still be used as an auxiliary variable for stratified sampling.
1
Sources
50%
Confidence
Long-term
Relevance
10/27/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIf monitoring covers only final business metrics, the small traffic sample in the canary phase may be insufficient to trigger a statistically significant alert, so more sensitive proxy metrics such as prediction distribution and feature distribution should also be compared, with reasonable sample sizes and confidence intervals set.78% similarUnverifiedWhen evaluating any statistical figure, one should ask about the denominator, whether 'don't know' respondents were excluded, whether the sample introduces systematic bias, and whether the headline framing matches the rigor of the underlying data.75% similarUnverifiedThe study recommends that benchmarks should be run multiple times and report variance intervals, explicitly marking which comparative conclusions are statistically inconclusive74% similarUnverifiedMost benchmarks use accuracy as the core metric, counting both wrong answers and refusals as incorrect and not distinguishing the difference in harm.74% similarUnverifiedWhen using reference-set-driven scoring, it usually needs to be paired with manual spot checks, or treated as an alignment check rather than a quality verdict.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1103042API
curl https://kongchang.com/api/v1/knowledge/claims/1103042MCP
get_claim(id=1103042)