Unverified50% confidenceFactExact time
One AI model's score on the MATH500 math test rose from 7.9% to 48.1%, an increase of about sixfold, stemming from modifying 0.016% of another model's weights.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
Related Entities
Related Claims
UnverifiedIn a math repair experiment, a small open-source model scored only 7.9% on the Math 500 test; Astra raised the score to 48.1% by locating and subtracting a direction within two layers (about 130,000 weights, an edit ratio of 0.016%), and its instruction-following ability also improved roughly twofold.77% similarUnverifiedThe benchmark improvement from 7.9% to 48.1% refers to the change in the model's accuracy on a certain math benchmark; 7.9% is close to random guessing, while 48.1% indicates the model has substantive problem-solving ability.77% similarUnverifiedOn math tasks, as the training answers become increasingly absurd, the proportion of misaligned responses soars from 4% to over 60%, while the model's general capabilities show almost no loss.74% similarUnverifiedScale AI的研究团队发现，部分模型在被污染的基准上的表现比在新编写的同等难度题目上高出10-15个百分点72% similarUnverifiedTop AI models' scores on standard benchmarks see significant leaps every 6–9 months.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1135434API
curl https://kongchang.com/api/v1/knowledge/claims/1135434MCP
get_claim(id=1135434)