GPQA Diamond
GPQA Diamond is a graduate-level question-answering benchmark and the most difficult subset within the GPQA (Graduate-Level Google-Proof Q&A) series. The benchmark covers natural science disciplines including biology, chemistry, and physics, with questions designed by domain experts. It is intended to evaluate the performance of large language models on high-difficulty professional reasoning tasks and serves as an important reference standard for measuring scientific understanding in frontier AI models.
Timeline (last 90 days)
On the GPQA Diamond scientific reasoning benchmark, the RSI inference engine ranks 1st among models under 500B parameters.
VIDRAFT achieved 93.43% in the sub-500B parameter category on GPQA Diamond.
The floor price, which tracks the cheapest model meeting the GPQA Diamond ≥70 threshold, fell from $0.113 to $0.090, while the spread between it and the flagship ceiling price widened from 36.6× to 57.1×.
GPQA Diamond is a multiple-choice question set targeting doctoral-level scientific knowledge, covering biology, chemistry, and physics, with domain experts averaging approximately 65% accuracy.
在全部 38 项基准中，Aplomb 1 在 8 项上拿到 5.3B 以内模型最高分，包括 MMLU-Pro、GPQA Diamond 和 BBH
The IQ3S tier scores 92.93 on GPQA Diamond (vs. BF16 baseline 91.92), 86.86 on LiveCodeBench V6 (vs. 87.43), with a three-task average of 93.26 vs. 93.12, and a file size less than one quarter of BF16 (approximately 22%).
GPQA Diamond is a benchmark testing graduate-level scientific reasoning ability, with ordinary human performance at approximately 35% and top closed-source models in the 60–75% range.
The evaluation covered five benchmarks: MMLU-Pro, GPQA Diamond, GSM8K, German MGSM, and IFEval.
The GPT-6 Astra leak claims a score of 98% on FrontierMath Tier 4.
The NVFP4 version of Qwen3-27B shows only marginal loss compared to uncompressed FP16 on the MMLU Pro and GPQA Diamond benchmarks.
2 more timeline events
All Facts (12)
GPQA Diamond测试题目来自生物学、化学、物理学等领域的博士级别知识，领域专家平均正确率约65%
85%UnverifiedGPQA Diamond由纽约大学研究团队构建，人类专家（博士生）的平均正确率约为65%，非专业人类约为34%
85%UnverifiedVIDRAFT achieved 93.43% in the sub-500B parameter category on GPQA Diamond.
60%UnverifiedQwen3.8-27B trails Claude Opus 4.6 on Terminal Bench, GPQA Diamond, and HLE benchmarks (73 vs 78.2, 89.2 vs 91.3, and 30.8 vs 40, respectively).
60%UnverifiedOn the GPQA Diamond scientific reasoning benchmark, the RSI inference engine ranks 1st among models under 500B parameters.
50%UnverifiedThe floor price, which tracks the cheapest model meeting the GPQA Diamond ≥70 threshold, fell from $0.113 to $0.090, while the spread between it and the flagship ceiling price widened from 36.6× to 57.1×.
50%UnverifiedGPQA Diamond is a multiple-choice question set targeting doctoral-level scientific knowledge, covering biology, chemistry, and physics, with domain experts averaging approximately 65% accuracy.
50%Unverified在全部 38 项基准中，Aplomb 1 在 8 项上拿到 5.3B 以内模型最高分，包括 MMLU-Pro、GPQA Diamond 和 BBH
50%UnverifiedThe IQ3S tier scores 92.93 on GPQA Diamond (vs. BF16 baseline 91.92), 86.86 on LiveCodeBench V6 (vs. 87.43), with a three-task average of 93.26 vs. 93.12, and a file size less than one quarter of BF16 (approximately 22%).
50%UnverifiedGPQA Diamond is a benchmark testing graduate-level scientific reasoning ability, with ordinary human performance at approximately 35% and top closed-source models in the 60–75% range.
50%UnverifiedThe evaluation covered five benchmarks: MMLU-Pro, GPQA Diamond, GSM8K, German MGSM, and IFEval.
50%UnverifiedThe NVFP4 version of Qwen3-27B shows only marginal loss compared to uncompressed FP16 on the MMLU Pro and GPQA Diamond benchmarks.
50%