BenchmarkMMLU

MMLU Pro

MMLU Pro（Massive Multitask Language Understanding Pro）是一个大规模多任务语言理解基准测试，是原版MMLU的增强版本。该基准涵盖数学、科学、工程、法律、人文等多个学科领域，通过更具挑战性的多项选择题评估大型语言模型的综合知识理解与推理能力。相较于原版MMLU，其题目难度更高、选项更多，旨在更精准地区分顶尖语言模型的性能差异。

Timeline (last 90 days)

Sep 11

In the MMLU Pro test, 8-bit narrowly beat OQ4E with 60.6% versus 59.9%.

Unverified50%

Sep 11

The NVFP4 version of Qwen3-27B shows only marginal loss compared to uncompressed FP16 on the MMLU Pro and GPQA Diamond benchmarks.

Unverified50%

All Facts (3)

Unverified

MMLU Pro涵盖57个学科领域，人类专家平均得分约为78分

75%Unverified

In the MMLU Pro test, 8-bit narrowly beat OQ4E with 60.6% versus 59.9%.

50%Unverified

The NVFP4 version of Qwen3-27B shows only marginal loss compared to uncompressed FP16 on the MMLU Pro and GPQA Diamond benchmarks.

50%

Source Articles

Gemini 3.5 Pro In-Depth Review: Dominant Multimodal Lead, Full Analysis of a 9.2-Rated FlagshipMay 23