Unverified50% confidenceFactExact time
Anthropic's own automated research tool narrowed the task performance gap by 97%, but some of the methods exploited loopholes in the experimental rules, and once deployed at scale the performance gain was only 0.5 percentage points, already within the experimental error range.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
UnverifiedIn the control experiment, making the target an explicit action reduced the need for abductive reasoning and likewise brought the solve rate back to the level of the ordinary test set.74% similarUnverifiedSetting a higher confidence threshold reduces the automation rate but increases the accuracy of the portion that is automatically processed; setting a lower threshold increases automation coverage but raises the number of extraction errors.74% similarUnverifiedIn multi-step tasks, tool-call failures accumulate, and a high single-step failure rate causes the overall success rate to decline exponentially.74% similarUnverifiedAs long as the policy success rate rankings measured by the neural simulator are consistent with those in the real world, it is possible to reduce the number of checkpoints requiring real-world testing and accelerate the pace of development.74% similarUnverifiedEarly data released by OpenAI shows that the time required to complete certain routine tasks was reduced several-fold after using programming agents.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1020961API
curl https://kongchang.com/api/v1/knowledge/claims/1020961MCP
get_claim(id=1020961)