Unverified50% confidenceOpinionExact time
A single call passing the test only gives an n=1 result, which is insufficient to prove model reliability; the pass rate and the lower bound of its confidence interval need to be computed over multiple trials.
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedA high score on any single benchmark test (such as 99.9%) only shows that the model performs excellently within the distribution of that test's questions, and cannot be directly equated with general intelligence.78% similarUnverified多重比较问题指同时测试多个模型变体时，即使每个单独检验显著性水平设为0.05，整体假阳性率也会急剧膨胀78% similarUnverifiedBenchmarks typically count only the cost of one successful call and completely ignore the repeated trial and error in real workflows; counting only the cost of one successful call is equivalent to assuming a retry rate of zero, which is severely disconnected from actual bills.77% similarUnverified对同一测试用例执行多次调用统计成功比例得到通过率，比二元的通过/失败判断更能反映模型真实表现77% similarUnverified用单一测试案例评估模型性能是一个统计陷阱，即小样本偏差（Small Sample Bias）74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1052528API
curl https://kongchang.com/api/v1/knowledge/claims/1052528MCP
get_claim(id=1052528)