Unverified50% confidenceBenchmarkExact time
In the discovery phase, 15 models were tested; the strongest-performing models were able to discover over 80% of the hidden requirements, while open-weight models could only discover 40-50%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/3/2026
First Seen
Valid until: 2/1/2027
Sources
Related Entities
Related Claims
UnverifiedOn two tasks, biomedical inference and vulnerability detection, tested against 18 models, the best-of-10 attack flipped correct answers for an average of 54.9% of cases on proprietary models and reached 80.0% on open-weight models.73% similarUnverifiedThe benchmark reveals significant differences among ten models across the three dimensions of robustness to omission, evidence acquisition, and appropriate action.68% similarUnverifiedIn a test that used Traework to fully simulate a product development workflow, the tool retrieved more than 100 sources during the research phase, but after manual verification only about 40 were truly usable, while the other 60 needed to be discarded or re-investigated.68% similarUnverifiedFrontier models are still stronger than open-source models, but the gap is narrowing; a reference ratio is to use about 20% frontier models for the most critical tasks and hand long-tail tokens to smaller models.67% similarUnverifiedIn the TRACES test, every model had a failure rate above 71% on the 'agentic' probes; 22 of the 30 models had failure rates above 90%.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1130180API
curl https://kongchang.com/api/v1/knowledge/claims/1130180MCP
get_claim(id=1130180)