Unverified50% confidenceSolutionExact time
When selecting cases, priority should go to examples near the decision boundary where a cheaper or differently tuned model is suspected of failing; cases that have never failed are only smoke tests, not regression tests.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn industrial inspection scenarios with almost no defect samples, anomaly detection should be prioritized over supervised learning, because what is available is a large volume of normal samples rather than defects.75% similarUnverifiedDuring evaluation, per-case experiment diffs should be examined rather than only looking at changes in aggregate scores, in order to pinpoint regressions to specific cases75% similarUnverifiedMost benchmarks use accuracy as the core metric, counting both wrong answers and refusals as incorrect and not distinguishing the difference in harm.72% similarUnverifiedMandatory checks should be separated from scoring; no matter how high the score on optional check items, it cannot offset a failed mandatory check.71% similarUnverifiedThe author calls the above benchmark a "smoke test, not a real-world accuracy claim," with a test sample of only 4 receipts (3 of which were used for benchmark evaluation).71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1066527API
curl https://kongchang.com/api/v1/knowledge/claims/1066527MCP
get_claim(id=1066527)