Unverified50% confidenceFactExact time
The variance analysis in the SWE-Bench Pro V2 paper shows that the scaffold can explain only about 0.29% of the result variance, while the model explains 2.65%.
1
Sources
50%
Confidence
Long-term
Relevance
10/24/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn the specific experimental case, the overlapping-anchor scheme (n=912) showed the model at 55.2% versus the majority class at 51.1%, a net advantage of +4.1pp, about 2.4 standard errors; the non-overlapping-anchor scheme (n=163) showed 59.5% versus 54.0%, a net advantage of +5.5pp, about 1.4 standard errors.68% similarUnverifiedCognition states that SWE-2 scores only one point below Anthropic's Claude model on its own coding benchmarks, yet costs 64% less68% similarUnverifiedOn Track B, the flat RAG baseline can detect 76%–97% of true contradictions, but produces false positives on 16%–43% of surface-similar safe drafts (the specific ratio depends on the backend).66% similarUnverifiedCompared with Transolver on the same dataset and the same data split, the model reduces relative L2 error by 31% and interior mean squared error (interior MSE) by 47%.66% similarUnverifiedDuring epochs 29 to 50, the variance of the sign-pattern loss is only ±0.40%, whereas the magnitude-based variant reaches as high as ±2.25%65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1094311API
curl https://kongchang.com/api/v1/knowledge/claims/1094311MCP
get_claim(id=1094311)