Unverified50% confidenceDecision RuleExact time
The author's criterion: a mutation pass rate below 70% indicates holes in the tests, meaning the probes did not reach far enough.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/3/2026
First Seen
Valid until: 2/1/2027
Sources
Are 90% of AI-Written Tests Pseudo-Logic? Three Blind Spots and Probe Tactics
bilibili知了知了啦10/25/2026
Related Entities
Related Claims
UnverifiedTop academic deepfake detectors can see their accuracy plummet from 99% to below 60% when faced with previously unseen generative models, a phenomenon known as the cross-model generalization problem.74% similarUnverifiedA detection rate of 73.43% means that approximately one in four core errors still goes uncaught, indicating that automated review cannot yet fully replace the judgment of human experts.74% similarUnverifiedEven when the agent can query the complete reference implementation, adding the skeleton still reduced average test error by up to 27.0% relative.74% similarUnverifiedIn a test of 119 pairs of "near-identical questions", at the 0.81 threshold 62% of modified questions incorrectly hit; even raised to 0.90, 25% still hit; the hit rate for genuine synonymous rewrites dropped from 39% to 4%.74% similarUnverifiedAn insufficiently small reference corpus systematically underestimates true divergence, and the researchers introduce a source-split jackknife method that uses only reference data, which can reduce calibration error by 57% to 59%.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129892API
curl https://kongchang.com/api/v1/knowledge/claims/1129892MCP
get_claim(id=1129892)