Unverified50% confidenceFactExact time
Two models may be comparable in exact-match accuracy, but on shared test instances the number of cells they get wrong may differ greatly.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/16/2026
First Seen
Valid until: 1/14/2027
Sources
Related Entities
Related Claims
Unverified对同一测试用例执行多次调用统计成功比例得到通过率，比二元的通过/失败判断更能反映模型真实表现78% similarUnverifiedThe statistical premise of drift detection is that the two sets of observations being compared must be comparable samples from the same distribution; directly comparing raw counts across versions produces false positives.78% similarUnverifiedTwo models with similar intelligence scores may differ in inference cost by several times or even an order of magnitude.76% similarUnverifiedMcNemar 检验关注两个模型在每个样本上的分歧情况，只有判断结果不一致的样本（b和c单元格）具有信息量，两者都对或都错的样本被视为无关信息75% similarUnverified多重比较问题指同时测试多个模型变体时，即使每个单独检验显著性水平设为0.05，整体假阳性率也会急剧膨胀74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1041954API
curl https://kongchang.com/api/v1/knowledge/claims/1041954MCP
get_claim(id=1041954)