Unverified50% confidenceBenchmarkExact time
In the τ-bench retail environment benchmark test (30 test tasks, simulated customers, single run, solvi 0.8.0, gpt-oss-120b as the Agent), without the guard 18 tasks were solved and the environment rejected 10 mutating calls; with the guard 14 tasks were solved and the environment rejected 0 mutating calls.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/12/2026
First Seen
Valid until: 10/26/2026
Sources
Related Entities
Related Claims
UnverifiedIn the experiments, the task resolution rate was 17% for zero-shot generation, 27% for generic self-correction, 23% for error-only feedback, and 90% for diagnostic counterexample feedback (A-CEGIS).65% similarUnverifiedIn the τ-bench retail probe, the action model predicted the environment's rejections with precision and recall of 1.000 on 11,763 held-out transitions, abstaining in only 0.26% of cases.64% similarUnverifiedIn a test that used Traework to fully simulate a product development workflow, the tool retrieved more than 100 sources during the research phase, but after manual verification only about 40 were truly usable, while the other 60 needed to be discarded or re-investigated.64% similarUnverifiedThe current dataset contains 770 recorded attempts across 29 model/configuration entries, but the unified test suite includes only 16 distinct tasks.63% similarUnverified在 Terminal Bench 2.1 测试中，OpenCode 对比里 89 个任务的失败数从 29 降到 16，多解决 13 题，分数提升 14.61 个百分点63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1006096API
curl https://kongchang.com/api/v1/knowledge/claims/1006096MCP
get_claim(id=1006096)