Unverified50% confidenceBenchmarkExact time
The same model (with identical weights) scores 62% in one agent harness, but only 33% in a different harness, a gap of nearly 30 percentage points.
1
Sources
50%
Confidence
Long-term
Relevance
10/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAfter matched supervised training, model performance improved to 99.8%, while removing the relational links caused accuracy to plummet to 23.4%, even below the best shortcut's 27.0%.63% similarUnverifiedSystems using the same or similar underlying models but with different harness designs can differ by more than 10–20 percentage points in fix success rate on SWE-bench.62% similarUnverifiedFor comparison, the Program-of-Thought baseline achieves 72.0% overall accuracy and the tool-calling agent achieves 58.7%.62% similarUnverifiedA single performance percentage is misleading; a model that claims to retain 98% of performance may happen to fall within the lost 2% for a specific task type, and real tasks should be tested before trusting benchmarks.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/980167API
curl https://kongchang.com/api/v1/knowledge/claims/980167MCP
get_claim(id=980167)