MGSM
A multilingual grade-school math reasoning benchmark dataset that translates GSM8K problems into multiple languages, used to evaluate the cross-lingual mathematical reasoning ability of large language models.
Timeline (last 90 days)
A new paper examined three key questions across four open-weight large language models using two mathematical reasoning datasets, MATH and MGSM: whether language subspaces capture translation-related variation, whether projection subspaces can remove language information, and whether accuracy can be improved without harming the response language.
On Qwen2.5-7B, the intervention that improved MGSM accuracy the most was precisely the one that sacrificed the fidelity of the response language.
The evaluation covered five benchmarks: MMLU-Pro, GPQA Diamond, GSM8K, German MGSM, and IFEval.
On GSM8K, Flash-Next scored 97.5% versus FP8's 90.0%; on German MGSM, 92.5% versus 90.0%; and on IFEval, both tied at 89.6%.
All Facts (3)
A new paper examined three key questions across four open-weight large language models using two mathematical reasoning datasets, MATH and MGSM: whether language subspaces capture translation-related variation, whether projection subspaces can remove language information, and whether accuracy can be improved without harming the response language.
50%UnverifiedOn Qwen2.5-7B, the intervention that improved MGSM accuracy the most was precisely the one that sacrificed the fidelity of the response language.
50%UnverifiedThe evaluation covered five benchmarks: MMLU-Pro, GPQA Diamond, GSM8K, German MGSM, and IFEval.
50%