Math 500
A mathematical reasoning benchmark containing 500 competition-level mathematics problems, used to evaluate the mathematical problem-solving ability of large language models.
Timeline (last 90 days)
In experiment 8, Astra fixed the "looping" problem of the Qwen 3.5B (800 million parameter) model by editing its weights, after which the Math 500 score rose from slightly below 8% to slightly above 48% and GSM8K rose from 13.8% to 67.1%.
In a math repair experiment, a small open-source model scored only 7.9% on the Math 500 test; Astra raised the score to 48.1% by locating and subtracting a direction within two layers (about 130,000 weights, an edit ratio of 0.016%), and its instruction-following ability also improved roughly twofold.
Without any training, Astra modified only 0.016% of the weights of a small model, raising its score on the Math 500 benchmark from 7.9% to 48.1%.
The Math 500 results were obtained under conditions that allowed the use of Python and actually running the networks on up to 50 batches of practice problems, not pure weight reading or zero-shot reasoning.
Math 500 is a benchmark containing 500 competition-style math problems, and answers are usually required to match exactly.
All Facts (5)
In experiment 8, Astra fixed the "looping" problem of the Qwen 3.5B (800 million parameter) model by editing its weights, after which the Math 500 score rose from slightly below 8% to slightly above 48% and GSM8K rose from 13.8% to 67.1%.
50%UnverifiedIn a math repair experiment, a small open-source model scored only 7.9% on the Math 500 test; Astra raised the score to 48.1% by locating and subtracting a direction within two layers (about 130,000 weights, an edit ratio of 0.016%), and its instruction-following ability also improved roughly twofold.
50%UnverifiedWithout any training, Astra modified only 0.016% of the weights of a small model, raising its score on the Math 500 benchmark from 7.9% to 48.1%.
50%UnverifiedThe Math 500 results were obtained under conditions that allowed the use of Python and actually running the networks on up to 50 batches of practice problems, not pure weight reading or zero-shot reasoning.
50%UnverifiedMath 500 is a benchmark containing 500 competition-style math problems, and answers are usually required to match exactly.
50%