ARC-AGI-3
An AI benchmark designed by François Chollet that evaluates a model's generalization and reasoning capabilities through visual pattern recognition tasks, explicitly targeting reasoning rather than memorization. It is regarded as a reference measure of how closely AI systems approach general intelligence. ARC-AGI-3 is the latest and most difficult version of the ARC-AGI benchmark series, introduced in 2025.
Core Facts
Timeline (last 90 days)
The poster noted that the leaderboard image in the post is somewhat outdated, and the specific figures may reflect delays or differences in how they are measured.
The 56% score on ARC-AGI-3 still falls considerably short of top human performance and a perfect score.
The driving force behind the surge in ARC-AGI-3 scores is breakthroughs in model harnesses and methodology, not computational scale.
Kaggle competitions impose strict restrictions on the models participants may use, allowing contestants to call only smaller local models.
Over the past 30 days, the top score on ARC-AGI-3 on the Kaggle platform climbed from 7% to 56%.
ARC-AGI-3 is a reasoning capability benchmark designed by François Chollet that tests AI's ability to generalize reasoning to task rules it has never seen before, specifically designed to resist brute-force memorization.
The energy consumed by a human player completing one round of ARC-AGI-3 costs less than $0.1 at retail electricity prices.
ARC-AGI-3 is the latest version of the ARC-AGI series, with further increased difficulty.
The cost for a system called Astra to complete the same round of ARC-AGI-3 is as high as $300 to $400.
Under the standard framework of ARC Prize, GPT-6 Astra's ARC-AGI-3 score is only 62.7.
18 more timeline events
All Facts (20)
Under the standard framework of ARC Prize, GPT-6 Astra's ARC-AGI-3 score is only 62.7.
65%VerifiedGPT-6 Astra achieved a score of 99.9% on the ARC-AGI 3 benchmark using OpenAI's custom Provider Adapter framework, at a cost of $19,000.
65%UnverifiedThe key upgrade in ARC-AGI-3 is the introduction of interactive reasoning, where agents need to gradually approach the correct solution through multi-round exploration, trial and error, and feedback in a dynamic environment.
60%UnverifiedClaude Opus 5 scored 30.2% on ARC-AGI-3, compared to only 1.5% for the previous generation.
60%UnverifiedThe poster noted that the leaderboard image in the post is somewhat outdated, and the specific figures may reflect delays or differences in how they are measured.
50%UnverifiedOver the past 30 days, the top score on ARC-AGI-3 on the Kaggle platform climbed from 7% to 56%.
50%UnverifiedKaggle competitions impose strict restrictions on the models participants may use, allowing contestants to call only smaller local models.
50%UnverifiedThe driving force behind the surge in ARC-AGI-3 scores is breakthroughs in model harnesses and methodology, not computational scale.
50%UnverifiedThe 56% score on ARC-AGI-3 still falls considerably short of top human performance and a perfect score.
50%UnverifiedARC-AGI-3 is a reasoning capability benchmark designed by François Chollet that tests AI's ability to generalize reasoning to task rules it has never seen before, specifically designed to resist brute-force memorization.
50%UnverifiedThe energy consumed by a human player completing one round of ARC-AGI-3 costs less than $0.1 at retail electricity prices.
50%UnverifiedARC-AGI-3 is the latest version of the ARC-AGI series, with further increased difficulty.
50%UnverifiedARC-AGI-3 is no longer limited to the static task format of input grid/output grid, but instead introduces dynamic environment testing more akin to agent behavior.
50%UnverifiedThe cautious camp believes that conquering ARC-AGI-3 may require fundamental breakthroughs at the architectural level rather than simply stacking parameter scale.
50%UnverifiedOn the ARC-AGI-3 benchmark, the same GPT-5.6 model with only two configuration items in the Codex Harness changed saw its score rise from 13.3% to 38.3%, with output tokens reduced by a factor of six.
50%UnverifiedThe interactive reasoning test requires agents to coordinate three capabilities: environmental understanding, plan formulation, and dynamic adjustment.
50%UnverifiedThe Chollet team continues to release new versions of ARC-AGI (such as ARC-AGI-2 and ARC-AGI-3) to increase difficulty and diversity.
50%UnverifiedThe core innovation of ARC-AGI-3 compared to its predecessor is the introduction of interactive reasoning.
50%UnverifiedIn ARC-AGI-3, systems can submit tentative answers and receive feedback, thereby progressively refining their solutions through multiple iterations.
50%Unverified在完全相同的模型下，仅靠Harness策略优化，ARC-AGI-3得分从13.3%提升到38.3%，同时输出Token节省约六倍
50%