Decision Index
A benchmark leaderboard established specifically for decision-oriented models covering classification, scoring, routing, and structured judgment tasks. Its evaluation metrics differ from those of general-purpose language model leaderboards, focusing on performance assessment for tasks with finite discrete outputs.
Timeline (last 90 days)
The community-run Decision Index leaderboard shows Perplexity at 62.75 and Jev at 60.11 (v0.3 scoring), but the scoring formula changes between versions, so the two numbers cannot be directly compared.
On the self-reported Cloudflare internal Decision Index 0.2.1 benchmark, clef-omni leads Jev by a wide margin on intent classification and routing tasks (BANKING77, CLINC150); but on tasks requiring multi-step reasoning (GPQA, BBH, MMLU-Pro), Jev leads by 12 to 31 points.
Decision Index is an evaluation benchmark designed specifically for decision-type models, focusing on structured judgment scenarios such as multi-attribute option comparison, risk probability estimation, and ranking consistency
Aplomb 1 在 Decision Index 0.2.1 上以官方工具跑出 44.86 分，位列已公布榜单中 4B 模型第一名
The author claims that TinyDecide is the smallest model on the Decision Index leaderboard.
Decision Index is a benchmark leaderboard specifically targeting decision-making tasks, and its evaluation metrics differ from those of general language model leaderboards such as MMLU and HellaSwag.
All Facts (6)
The community-run Decision Index leaderboard shows Perplexity at 62.75 and Jev at 60.11 (v0.3 scoring), but the scoring formula changes between versions, so the two numbers cannot be directly compared.
50%UnverifiedOn the self-reported Cloudflare internal Decision Index 0.2.1 benchmark, clef-omni leads Jev by a wide margin on intent classification and routing tasks (BANKING77, CLINC150); but on tasks requiring multi-step reasoning (GPQA, BBH, MMLU-Pro), Jev leads by 12 to 31 points.
50%UnverifiedDecision Index is an evaluation benchmark designed specifically for decision-type models, focusing on structured judgment scenarios such as multi-attribute option comparison, risk probability estimation, and ranking consistency
50%UnverifiedAplomb 1 在 Decision Index 0.2.1 上以官方工具跑出 44.86 分，位列已公布榜单中 4B 模型第一名
50%UnverifiedThe author claims that TinyDecide is the smallest model on the Decision Index leaderboard.
50%UnverifiedDecision Index is a benchmark leaderboard specifically targeting decision-making tasks, and its evaluation metrics differ from those of general language model leaderboards such as MMLU and HellaSwag.
50%