Unverified50% confidenceBenchmarkExact time
After using the distilled artifact, the spread in success rates across the four models narrowed from 21.6 percentage points to 3.9 percentage points, and the average success rate of all four models exceeded 90%.
1
Sources
50%
Confidence
Long-term
Relevance
11/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn tasks requiring more than 5 steps, the success rate of the strongest models drops sharply from over 80% at the first step to 40% or even lower at subsequent steps, exhibiting an exponential decline.78% similarUnverifiedAssuming the agent has a 90% correctness rate per step, chaining 3 steps yields a 73% success rate (0.9^3), 5 steps 59% (0.9^5), and 10 steps 35% (0.9^10).76% similarUnverifiedAfter enabling deterministic mode, performance typically drops by 5%-20%, depending on the model and operator combination.74% similarUnverifiedAcross the four evaluated models, refined artifacts raised the average reproduction success rate from 76.4% to 92.6%.74% similarVerifiedIn a 10-step engineering task where each step has a 90% success rate, the overall task completion rate is approximately 35%; if each step's success rate is raised to 99%, the overall success rate jumps to 90%.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1142644API
curl https://kongchang.com/api/v1/knowledge/claims/1142644MCP
get_claim(id=1142644)