Vals AI
A third-party AI benchmarking company that positions itself as a neutral and trustworthy evaluation institution. Vals AI provides standardized and transparent model assessment services aimed at enterprise users and developers seeking independent appraisals of AI model capabilities.
Timeline (last 90 days)
Vals AI released its latest benchmark report at the AI Engineer NYC conference.
Vals AI is an organization focused on independent evaluation of AI models, and its reports are often cited by the developer community as a basis for horizontal comparison.
AI evaluation standards are shifting from academically oriented to product-oriented, and the test sets of CursorBench and Vals AI are both closer to real development workflows.
Vals Index is a comprehensive index launched by the independent evaluation organization Vals AI, which weights and aggregates results from multiple task categories such as finance, programming, law and tax into a single score, without relying on vendor self-reported data.
According to testing by Artificial Analysis and Vals AI, Gemini 4 Argon outputs about 62k tokens per task at the high level, while GPT-6 Astra outputs about 27k tokens at the max level.
All public benchmarks run Gemini 4 Argon at the thinking_level=high level, including evaluations by Google, Artificial Analysis, and Vals AI.
Vals AI uses benchmarks focused on real productized scenarios and agentic tasks, including long-chain benchmarks such as CUA-bench, Code Migration, Terminal-Bench Science, and SRE Bench.
Per-task cost is highly dependent on the specific workload, and different combinations of benchmarks and workloads may yield opposite cost conclusions.
Vals AI's independent leaderboard places Gemini 4 Argon first among 41 models (68.9%).
Vals measured Gemini 4 Argon's cost per test at $15.68, lower than Claude Sonnet 5.5's $21.34 and Opus 5.5's $32.14, making it the lowest in the group.
6 more timeline events
All Facts (14)
AI evaluation standards are shifting from academically oriented to product-oriented, and the test sets of CursorBench and Vals AI are both closer to real development workflows.
50%UnverifiedVals AI is an organization focused on independent evaluation of AI models, and its reports are often cited by the developer community as a basis for horizontal comparison.
50%UnverifiedVals Index is a comprehensive index launched by the independent evaluation organization Vals AI, which weights and aggregates results from multiple task categories such as finance, programming, law and tax into a single score, without relying on vendor self-reported data.
50%UnverifiedAll public benchmarks run Gemini 4 Argon at the thinking_level=high level, including evaluations by Google, Artificial Analysis, and Vals AI.
50%UnverifiedAccording to testing by Artificial Analysis and Vals AI, Gemini 4 Argon outputs about 62k tokens per task at the high level, while GPT-6 Astra outputs about 27k tokens at the max level.
50%UnverifiedVals AI uses benchmarks focused on real productized scenarios and agentic tasks, including long-chain benchmarks such as CUA-bench, Code Migration, Terminal-Bench Science, and SRE Bench.
50%UnverifiedVals AI's independent leaderboard places Gemini 4 Argon first among 41 models (68.9%).
50%UnverifiedPer-task cost is highly dependent on the specific workload, and different combinations of benchmarks and workloads may yield opposite cost conclusions.
50%UnverifiedVals measured Gemini 4 Argon's cost per test at $15.68, lower than Claude Sonnet 5.5's $21.34 and Opus 5.5's $32.14, making it the lowest in the group.
50%UnverifiedIn VALS AI's composite ranking, the VALS Index, Argon ranks first among 41 models.
50%UnverifiedVals AI hopes to become the gold standard in AI benchmarking, providing neutral and credible third-party evaluation resources.
50%UnverifiedThe title of the Vals AI official blog post is "Fable Solves Cyphral Distich."
50%UnverifiedVals AI released its latest benchmark report at the AI Engineer NYC conference.
50%UnverifiedVals AI has received investment from Andreessen Horowitz (a16z).
50%