180B Model Tops MMLU-Pro and GPQA: Interpreting the Darwin-180B-RSI Benchmark

VIDRAFT's open-source model Darwin-180B-RSI, a 180B-parameter model from the South Korean AI startup, recently took first place on both MMLU-Pro and GPQA Diamond on AIhoster, a Russian independent benchmark aggregator. With only 180B parameters, it beat out the 1.6T DeepSeek-V4-Pro and the 2.8T Kimi-K3 — a result worth unpacking.

What the model and the leaderboard are

Darwin-180B-RSI is a 180-billion-parameter open-weight language model developed by VIDRAFT (비드래프트), published under the FINAL-Bench/Darwin-180B-RSI repository on Hugging Face. VIDRAFT positions itself as an AI company in the "Pre-AGI" stage, and the Darwin series is its flagship product line.

AIhoster, meanwhile, is an independent benchmark aggregator whose approach differs from recomputation platforms like LMArena: it collects scores reported on model cards, papers, and public leaderboards rather than re-running evaluations or building normalized composite indices. Each record is tied to an exact repository ID and evaluation configuration, which is more transparent methodologically — but it also means the credibility of the scores depends on whoever originally reported them.

As of the October 19, 2026 update, Darwin-180B-RSI was evaluated on all 12,032 questions in MMLU-Pro and all 198 questions in GPQA Diamond, ranking first on both.

What the evaluation configuration reveals

The publicly disclosed evaluation configuration contains several key details:

• A thinking budget of up to 131,072 tokens. Both benchmarks used this setting, indicating that the model follows a long-chain reasoning approach, allocating a large number of tokens to intermediate derivation before producing a final answer.
• Sampling strategy differs by benchmark. MMLU-Pro used single sampling (temperature=1.0, top_p=0.95, top_k=20), while GPQA Diamond used majority voting over up to 16 samples with the same parameters — a classic case of test-time compute scaling, trading inference cost for accuracy.
• Precision is bf16, the standard choice for large-scale inference.

Architecture details, training hyperparameters, and dataset information were not disclosed. Working backward from the results, reaching this level with 180B parameters most likely comes from optimizations in architecture choices or training data curation rather than simply scaling up.

Thinking budget refers to the maximum number of tokens a reasoning model is allowed to use for internal intermediate reasoning before generating a final answer. This mechanism was first popularized by the OpenAI o1 series and later became widespread in open-source reasoning models: the model is trained to first output a chain of thought and then give a final response. The larger the budget, the longer the reasoning paths the model can explore, which typically yields better performance on tasks requiring multi-step derivation such as math and science — but latency and cost rise linearly as well. A budget of 131,072 tokens means the intermediate reasoning for a single request can run to tens of thousands of words, far exceeding the output length of ordinary chat models.

Test-time compute scaling refers to improving accuracy by increasing compute investment during the inference phase while model parameters remain fixed. Common techniques include sampling multiple times and voting, extending the chain of thought, and search-based decoding. It complements training-time scaling (stacking parameters and data): the former spends money on the deployment side to buy scores, the latter spends money on the training side to buy capability. Majority voting is its most straightforward implementation — sampling the same question multiple times and taking the most frequent answer as the final output, on the premise that errors from independent sampling are random rather than systematic.

Score comparison on the two leaderboards

MMLU-Pro (multi-domain expert knowledge):

Rank	Model	Score	Parameters
1	Darwin-180B-RSI	88.1%	180B
2	MiniMax-M2.1	88.0%	229B
3	Intern-S2-Preview	88.0%	36.1B
4	Qwen3.5-397B-A17B	87.8%	403B
5	DeepSeekAd-V4-Pro	87.5%	1.6T

GPQA Diamond (expert-level physics, chemistry, biology):

Rank	Model	Score	Parameters
1	Darwin-180B-RSI	94.4%	180B
2	Kimi-K3	93.5%	2.8T
3	Darwin-397B-ZTC	93.4%	404B
4	Ornith-1.5-397B	92.8%	403B
5	Qwen3.8-2.4T-A95B	92.6%	2.4T

The top five on MMLU-Pro are separated by only 0.6 percentage points, meaning the first and fifth places are essentially in the same tier, and the top spot is quite sensitive to sampling randomness. The gap on GPQA is slightly larger, but here too the differences in inference conditions need to be considered: Darwin-180B-RSI's 94.4% comes from majority voting over up to 16 samples, whereas third-place Darwin-397B-ZTC's 93.4% uses greedy decoding with no voting. The two are not operating under the same inference budget, so directly comparing their scores is not fair.

Another noteworthy signal is VIDRAFT's product matrix. Three Darwin models appear on the GPQA leaderboard at once — 180B-RSI in first, 397B-ZTC in third, and 398B-JGOS in ninth (90.9%) — indicating a development pipeline that iterates in parallel across multiple scales and inference configurations, rather than a single-point breakthrough.

MMLU-Pro is an enhanced version of MMLU, introduced by TIGER-Lab in 2024. It changes the original four-choice format to ten-choice, adds more reasoning-intensive questions, and broadens subject coverage, totaling about 12,000 questions. With options increased from 4 to 10, the random-guessing baseline drops from 25% to 10%, which better distinguishes a model's true knowledge level and also discourages scoring points by process of elimination. It is also one of the most commonly cited knowledge benchmarks when open-source models report their results.

GPQA Diamond is the hardest and most rigorously filtered subset of GPQA (Graduate-Level Google-Proof Q&A), comprising 198 questions written by PhD-level question writers in physics, chemistry, and biology. It is designed so that even with search engines allowed, non-experts would struggle to answer correctly in a short time. Questions are four-choice with a random baseline of about 25%, but in practice even human experts (outside the relevant field) score noticeably lower than PhDs in the field. It is widely used to measure a model's reasoning ability on expert-level scientific questions; the small question count gives each question high weight, so sampling strategy has a particularly pronounced effect on the final score.

Efficiency advantages and open questions

Surpassing the 1.6T DeepSeek-V4-Pro on MMLU-Pro and the 2.8T Kimi-K3 on GPQA with 180B parameters represents a clear gain in parameter efficiency. At a time when inference cost has become a deployment bottleneck, efficiency differences of this magnitude have direct implications for real-world adoption.

But there are several questions the source itself does not answer:

Have the scores been independently reproduced? AIhoster explicitly states that it does not recompute scores, and the source of Darwin-180B-RSI's scores is its own model card. Third-party independent reproduction has not been confirmed.

Can it be used commercially? The license terms are not specified in the original materials; consult the Hugging Face model card before use.

Are the weights truly open? The materials only provide the repository path and a download command. Actual availability, API endpoints, and inference service availability are all unconfirmed.

huggingface-cli download FINAL-Bench/Darwin-180B-RSI

Open weights and open source are not legally or technically equivalent. Open weights only means the model parameter files can be downloaded; the training data, training code, and hyperparameter recipes are not necessarily public, and the license may impose commercial restrictions or user-scale thresholds (such as the custom community licenses adopted by some models). The article's question of whether the weights are "truly open" actually encompasses three verifiable issues: whether the parameter files can be downloaded reliably, whether the license permits the intended use, and whether usable inference services or quantized versions are provided. Missing any one of these affects the model's usability in production environments. When evaluating such models, the existence of a repository ID only proves that it was published; it cannot substitute for checking the license fields on the model card and the actual download availability.

What it means for developers

If the weights are indeed open and the license permits it, Darwin-180B-RSI is an option worth evaluating in scenarios that require frontier reasoning capability but are constrained by VRAM and inference budget. A 131K thinking budget combined with majority voting may give it an edge over models of comparable scale on hard math and science problems.

But topping a leaderboard does not equal production readiness. Scores sourced from the model card, an undisclosed architecture, an unspecified license, and the absence of third-party reproduction form a checklist that needs to be verified item by item. VIDRAFT's approach of advancing multiple Darwin variants simultaneously also means the current version may soon be superseded by an iterated release — following subsequent updates is more valuable than fixating on a single first-place finish.

One-line Summary

VIDRAFT's 180B open-source model Darwin-180B-RSI took first place on both MMLU-Pro and GPQA Diamond on the AIhoster leaderboard, but the scores are self-reported and have not been independently reproduced.

Paragraph Summary

The open-source model Darwin-180B-RSI from South Korean AI startup VIDRAFT took first place on both MMLU-Pro (88.1%) and GPQA Diamond (94.4%) on the independent benchmark aggregator AIhoster. With only 180B parameters, it surpassed the 1.6T DeepSeek-V4-Pro and 2.8T Kimi-K3. The evaluation configuration shows a thinking budget of up to 131,072 tokens, with the GPQA score obtained via majority voting over up to 16 samples. However, the top five on MMLU-Pro are separated by only 0.6 percentage points, and the GPQA comparison models use greedy decoding, so the inference budgets are not equivalent. AIhoster does not recompute scores — the results are self-reported on the model card — and third-party reproduction, license terms, and actual weight availability remain unconfirmed; topping a leaderboard does not equal production readiness.