Unverified60% confidenceFactExact time
If both models pass every case, then no matter how large the test set is, it provides no information about the difference between them.
2
Sources
60%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTwo models may be comparable in exact-match accuracy, but on shared test instances the number of cells they get wrong may differ greatly.74% similarUnverifiedMcNemar 检验关注两个模型在每个样本上的分歧情况，只有判断结果不一致的样本（b和c单元格）具有信息量，两者都对或都错的样本被视为无关信息73% similarUnverifiedWhen the 95% confidence intervals of two models overlap, the ranking difference between them may not be statistically meaningful.69% similarUnverifiedTwo samples are insufficient to estimate the flakiness rate; only the fact that the two results differed once is detected; the parent blob rule also does not evaluate assertion strength69% similarUnverifiedClustered standard errors target the situation of non-independent samples and can handle correlation between samples, whereas treating each sample as an independent observation underestimates the variance.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1066526API
curl https://kongchang.com/api/v1/knowledge/claims/1066526MCP
get_claim(id=1066526)