MATH-500
A mathematical reasoning benchmark dataset available on HuggingFace, consisting of 500 mathematics problems. It is used to evaluate the mathematical reasoning capabilities of large language models.
Timeline (last 90 days)
ZY-S1's MATH-500 dropping from 4 to 0 is a typical manifestation of catastrophic forgetting caused by a distribution shift in the fine-tuning data.
In the author's published 25-question small-sample evaluation, ZY-S1 dropped from 4 to 0 on MATH-500 compared with the base model, showing degradation in mathematical ability.
Without changing the prompt, the capabilities of the SymCE-trained model transfer to mainstream mathematical reasoning benchmarks such as GSM8K, MATH-500, and MMLU-college-math.
The study's experiments were conducted on 386 tasks from the MATH-500 dataset.
The experiment has a small sample size (single model, single test, 50 questions), is subject to random fluctuation, and constitutes a heuristic finding rather than a definitive conclusion.
A Reddit user randomly sampled 50 questions from the MATH-500 dataset and ran experiments across multiple quantized versions of Qwen3.5-4B.
AdaptiveSpec recovers 93% to fully lossless task accuracy on three major benchmarks: GSM8K, MATH-500, and HumanEval.
All Facts (7)
ZY-S1's MATH-500 dropping from 4 to 0 is a typical manifestation of catastrophic forgetting caused by a distribution shift in the fine-tuning data.
50%UnverifiedIn the author's published 25-question small-sample evaluation, ZY-S1 dropped from 4 to 0 on MATH-500 compared with the base model, showing degradation in mathematical ability.
50%UnverifiedWithout changing the prompt, the capabilities of the SymCE-trained model transfer to mainstream mathematical reasoning benchmarks such as GSM8K, MATH-500, and MMLU-college-math.
50%UnverifiedThe study's experiments were conducted on 386 tasks from the MATH-500 dataset.
50%UnverifiedThe experiment has a small sample size (single model, single test, 50 questions), is subject to random fluctuation, and constitutes a heuristic finding rather than a definitive conclusion.
50%UnverifiedAdaptiveSpec recovers 93% to fully lossless task accuracy on three major benchmarks: GSM8K, MATH-500, and HumanEval.
50%UnverifiedA Reddit user randomly sampled 50 questions from the MATH-500 dataset and ran experiments across multiple quantized versions of Qwen3.5-4B.
50%