Unverified50% confidenceBenchmarkExact time
Under adversarial perturbations simulating conflicting TTP statements, cross-report contamination, and semantic poisoning, ARBITRA limits average precision degradation to about half that of the frontier-model baselines.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/30/2026
First Seen
Valid until: 1/28/2027
Sources
Related Entities
Related Claims
UnverifiedA study published on arXiv (arXiv:2609.02899) proposes that concerns about benchmark contamination should be distinguished into two separate questions: whether absolute scores are inflated and whether the ranking order of models is changed.66% similarUnverifiedIn settings where the test risk is definable, the naive threshold-selection baseline exceeds the target risk level in about half of the cases, whereas the Learn Then Test (LTT) method exceeds it in only 7.8% of the cases.64% similarUnverifiedOn two tasks, biomedical inference and vulnerability detection, tested against 18 models, the best-of-10 attack flipped correct answers for an average of 54.9% of cases on proprietary models and reached 80.0% on open-weight models.64% similarUnverified卡内基梅隆大学研究显示对抗性图案攻击成功率在受控环境为90%以上，但引入正常视角变化后降至实际部署场景的30%以下64% similarUnverifiedOn a benchmark of 23 test scenarios and two tools, the strategy of "intercept on any change to the full set" had a false positive rate of 50%.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1113895API
curl https://kongchang.com/api/v1/knowledge/claims/1113895MCP
get_claim(id=1113895)