Hugging Face Official Benchmark Leaderboard Aggregator: Who's Leading Across 49 Leaderboards

On Hugging Face, when a dataset is officially marked as an "official benchmark," its page automatically generates a leaderboard — with data pulled from the .eval_results files published by model repositories. GPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, Terminal-Bench, MMMU-Pro, LEXam, and Open ASR Leaderboard all fall into this category.

The problem is that these leaderboards are scattered across dozens of independent dataset pages. There's no single place to answer a simple question: across all official benchmarks, who's actually ranked first?

Someone built an aggregator to answer this question.

What This Index Shows

It's a read-only, real-time view that recalculates from the Hugging Face API on every visit. The core dimensions include:

• Number of first-place positions by organization — how many official leaderboards each organization currently leads, with a share chart
• Number of first-place positions by country — first-place standings grouped by the country of the publishing organization
• Leaders by domain — a heatmap covering mathematics, science and knowledge, coding, agents and terminals, documents and OCR, vision and video, speech, law and finance, retrieval and robotics, and more
• First-place lead margin — the relative gap between first and second place on each leaderboard, which is direction-aware, so "lower is better" metrics like word error rate (WER) are handled correctly
• Per-benchmark details — the first and second place models and scores for each leaderboard, plus the number of entries

The value of this aggregation is that it turns a scattered information source into a comparable, structured view. In the past, determining whether an organization was "leading" required manually digging through dozens of pages; now you can see the distribution at a glance.

How It's Built

No data is entered by hand. The page reads the list of datasets marked as official benchmarks by Hugging Face, pulls ranking entries for each leaderboard from the public API, and then aggregates them. Since new first-place results only appear after Hugging Face has indexed the evaluation results, this index has the same latency as the source leaderboards.

A few design details are worth noting:

• Tied first places count toward all tied organizations simultaneously and aren't artificially split
• The first-to-second gap is measured relative to the second-place score and is direction-aware
• Country information comes from the publishing organization, not the model name — meaning the same model published by teams in different countries will be attributed to different countries

These design choices seem simple, but each one affects how the final statistics should be interpreted. Country attribution in particular is tricky: model names usually don't contain geographic information, and while the publishing organization's location is relatively more reliable, it still can't be fully equated with "which country the model came from."

.eval_results and the Official Benchmark Flag

Hugging Face model repositories allow publishers to upload a .eval_results directory containing a set of structured files (usually YAML or JSON) that record a model's score on a given benchmark, the metric name, evaluation time, and more. If a dataset page is officially marked as an "official benchmark," the system scans these files and automatically compiles matching results for that benchmark into a leaderboard. Understanding this is crucial: the numbers on the leaderboard aren't run by Hugging Face itself — they're declarations voluntarily submitted by model publishers. Verifiability therefore depends on whether the upload records are public and whether the timing is checkable. The aggregator works along this same chain — it reuses the same official flag and .eval_results data rather than building its own evaluation from scratch. This also explains why new results only appear on the aggregation page after Hugging Face has indexed them first.

Current Data Snapshot

As of early October 2026, the index tracks approximately 49 official benchmarks, totaling around one thousand leaderboard entries.

The single organization holding the most first-place positions is FINAL-Bench, with 11 top spots, covering areas including science, mathematics, multimodal, law, document extraction, structured output, and decision-making. This distribution is itself notable — it shows that leadership isn't concentrated in the traditional "large model" track, but is dispersed across multiple vertical capability dimensions.

Why This Matters

Benchmark claims are easy to make but difficult to verify. "We're leading on X" is one sentence, but for a reader to verify it, they typically need to find the corresponding dataset page, confirm the leaderboard exists, check when the evaluation results were submitted, and then compare against competing models' scores.

An aggregation page that reads directly from evaluation results turns that sentence into something verifiable with a single click — including exactly how large the lead is.

For teams, this is also a quick way to locate where the frontier is in your field and who's holding that position. When the lead margin is small, competition is fierce; when the gap is large, some model or method has a substantial advantage in that dimension.

The index itself doesn't make judgments — it just organizes public records into a comparable form. But for anyone who needs to track the landscape of model capabilities, that organization is itself a valuable step.

Live index URL: https://huggingface.co/spaces/quantid/huggingface-official-benchmark-leaderboards