whatmodel.sh
An open-source AI model leaderboard built by independent developer Jake Manger. It aggregates a large number of benchmark results and distinguishes independent evaluations from lab-reported ones. The dataset is licensed under CC BY-SA 4.0 and the scoring code under the MIT license, with everything open-sourced on GitHub.
Timeline (last 90 days)
The whatmodel.sh leaderboard built by developer Jake Manger shows that the capabilities of Chinese open-weight models lag about 7 months behind US closed-source models.
The whatmodel.sh leaderboard collected 22,113 benchmark results from 1,011 sources, covering 987 benchmarks.
The text model section of whatmodel.sh contains 18,755 results for 1,211 models on 617 benchmarks.
In the whatmodel.sh data, 14,436 results come from independent evaluations (ACL Anthology, arXiv, Epoch AI, LMArena, EvalPlus, EleutherAI harness, etc.) and 7,677 from lab self-reports, with independent evaluations accounting for about 77%.
The whatmodel.sh dataset uses the CC BY-SA 4.0 license, and the scoring code uses the MIT license, all open-sourced on GitHub.
All Facts (5)
The whatmodel.sh leaderboard built by developer Jake Manger shows that the capabilities of Chinese open-weight models lag about 7 months behind US closed-source models.
50%UnverifiedThe whatmodel.sh leaderboard collected 22,113 benchmark results from 1,011 sources, covering 987 benchmarks.
50%UnverifiedThe text model section of whatmodel.sh contains 18,755 results for 1,211 models on 617 benchmarks.
50%UnverifiedIn the whatmodel.sh data, 14,436 results come from independent evaluations (ACL Anthology, arXiv, Epoch AI, LMArena, EvalPlus, EleutherAI harness, etc.) and 7,677 from lab self-reports, with independent evaluations accounting for about 77%.
50%UnverifiedThe whatmodel.sh dataset uses the CC BY-SA 4.0 license, and the scoring code uses the MIT license, all open-sourced on GitHub.
50%