ARC-AGI-3 Nearly Solved: How Coding Ability Unlocks General AI Intelligence

A Tweet That Sparked Deep Reflection

Recently, a tweet about the ARC-AGI-3 benchmark sparked widespread discussion in the AI community. The tweet stated: simply by adding a "coding harness" to a large language model (LLM), ARC-AGI-3 was nearly solved. The author further emphasized that this validates an earlier prediction — coding ability helps LLMs achieve capability generalization (coding generalizes LLMs).

Though brief, this observation touches on one of the most central questions in current AI research: How do large models truly acquire general reasoning capabilities? And what critical role does code play in this process?

What Is ARC-AGI, and Why Does It Matter

A Litmus Test Designed for General Intelligence

ARC (Abstraction and Reasoning Corpus) was proposed by deep learning researcher François Chollet, designed specifically to measure AI systems' abstraction and reasoning abilities rather than memorization. Chollet is the creator of the Keras deep learning framework and a senior researcher at Google. In 2019, he published the highly influential paper On the Measure of Intelligence, systematically critiquing the limitations of AI benchmarks at the time — which tended to measure only a model's skill level on specific tasks rather than genuine intelligence. Chollet proposed that intelligence should be defined as "generalization efficiency in the face of novel situations" — a system's ability to adapt to entirely new problems using limited experience. ARC was designed based on this theoretical framework.

Unlike traditional benchmarks, ARC tasks are relatively intuitive for humans — inferring hidden transformation rules by observing a few input-output examples — but extremely challenging for AI systems. Specifically, each ARC problem consists of several "demonstration pairs" and a "test input." The demonstration pairs show a visual transformation rule — such as rotating, mirroring, filling, counting, or reorganizing regions of specific colors in a grid according to some topological relationship. The test-taker must infer the underlying rule from just these few demonstrations and apply it to the test input to generate the correct output. The grids are typically small (maximum 30×30) with a limited color palette (10 colors), but the diversity of transformation rules is virtually infinite. Humans can solve approximately 85% of ARC tasks on average, while even the strongest LLMs without special framework assistance have long plateaued at much lower scores.

The ARC-AGI series is considered an important yardstick for testing whether models possess "human-like generalization ability." For a long time, LLMs relying purely on pattern matching and massive data training performed poorly on such tasks because they require genuine reasoning on problems never seen before, rather than reproducing existing solutions from training data. Each problem is unique — there are no problem types that can be "gamed" through extensive practice — thereby maximally eliminating the influence of memorization and pattern matching.

The Evolution from ARC-AGI-1 to ARC-AGI-3

As model capabilities have improved, the ARC benchmark has undergone continuous iterative upgrades with increasing difficulty. ARC Prize is a competition co-founded by Chollet aimed at driving breakthroughs in general AI reasoning ability. The 2024 ARC Prize offered $1 million to any solution achieving 85% accuracy (human level) on ARC-AGI. In the competition's history, early top solutions relied heavily on domain-specific language (DSL) search — predefining a set of basic operations (such as rotation, translation, color replacement, etc.) and then searching combinations of these operations to fit the demonstration pairs. Later, hybrid approaches incorporating LLMs began to dominate, particularly methods that have LLMs generate candidate programs which are then verified through search.

ARC-AGI-3, as the latest generation of the test, was designed to counter powerful models capable of "gaming" the benchmark, pushing further toward testing genuine general reasoning ability. The introduction of ARC-AGI-2 and ARC-AGI-3 was specifically in response to increasingly powerful solutions, ensuring the benchmark can still effectively distinguish true reasoning ability from clever engineering optimization. Therefore, if the claim that "ARC-AGI-3 is nearly solved" holds up, its significance cannot be understated.

Why a Coding Harness Enables Reasoning Breakthroughs

Code as Scaffolding for Reasoning

The core insight here lies in the introduction of a "coding harness." A coding harness means the LLM no longer directly "guesses" answers but instead writes code to solve problems — transforming abstract reasoning tasks into executable, verifiable program logic.

Specifically, the core idea of a coding harness is to have the LLM generate Python programs describing the transformation logic from input to output, rather than directly predicting pixel values of the output grid. The typical workflow includes: the model observes input-output examples, analyzes possible transformation rules, then writes a transform function; this function is actually executed on the demonstration inputs to check whether the output matches the demonstration answers; if not, the model receives error feedback and iteratively modifies the code. The key advantage of this approach is introducing an "execution-verification loop" — code either runs correctly producing expected results or throws errors; there is no ambiguous middle ground. This makes the reasoning process debuggable and traceable.

The value of this approach manifests on several levels:

• Structured reasoning: Code inherently demands logical rigor and clear steps. When models are guided to think in a programming manner, they must transform vague intuitions into precise operational sequences.
• Verifiability: Programs can be executed and results checked; models can iteratively correct based on feedback rather than producing one-shot outputs.
• Compositional generalization: Programming is fundamentally about composing simple operations into complex logic — which aligns precisely with ARC tasks' requirement to "infer general rules from few examples."

The Deeper Logic of Code Generalizing LLM Capabilities

The tweet's statement "as predicted, coding generalizes LLMs" echoes a view gaining increasing consensus in AI research: training on code data and guiding reasoning through programming can significantly enhance a model's general capabilities on non-programming tasks.

Recent research has provided ample empirical support for this view. Research papers from institutions like Meta have shown that even for pure natural language tasks (such as mathematical reasoning and logical inference), models pretrained on code outperform text-only models. The reason is that code possesses inherent formalization characteristics: variable definitions require concept clarification, control flow requires explicit logic, and function abstraction requires problem decomposition. Furthermore, code's "correctness verifiability" property provides natural reward signals for training, opening new alignment paths beyond reinforcement learning from human feedback (RLHF). Institutions including OpenAI and Google DeepMind have all come to view coding ability as a key component on the path to AGI.

The underlying logic is that code is not merely a skill but a vehicle for modes of thinking. Writing code requires abstraction, decomposition, recursion, conditional reasoning, and other abilities — which are precisely the foundational building blocks of general reasoning. Therefore, when a model masters the approach of "thinking in code," its performance across broad domains like mathematics, logic, and planning improves accordingly.

The Paradigm Evolution from Tool Use to Program Synthesis

The combination of LLMs and code has progressed through several clear stages. Initially came "tool use," such as having models call calculators or search engines to compensate for their computational shortcomings; this then developed into "code interpreters," where models could write and execute code for data analysis, file operations, and other tasks — OpenAI's Code Interpreter feature was the landmark product of this stage; now it has further evolved into "program synthesis," where models generate programs to express and verify their reasoning processes.

The core shift in this paradigm is that reasoning is no longer a black-box process inside the model but is "externalized" into inspectable programs. This not only improves accuracy but also greatly enhances interpretability — we can directly read the model's generated code to understand its reasoning logic. The breakthrough on ARC-AGI-3 is the latest achievement of this paradigm evolution.

What This Discovery Means for AGI Research

A Possible Path Toward Artificial General Intelligence

If the conclusion that "adding a coding harness nearly solves ARC-AGI-3" withstands rigorous scrutiny, it sends an important signal: general reasoning ability may not require entirely new model architectures but can be elicited from existing LLMs through appropriate "guidance methods."

This aligns with the hypothesis that "models already possess latent reasoning capabilities and simply need the right framework to unlock them." In other words, the breakthrough may not lie in making models larger but in designing smarter reasoning and verification mechanisms — letting models "program first, execute second, correct third." This approach of "externalized thinking" is in some ways analogous to how humans use pen and paper, diagrams, and other external tools to assist complex reasoning — a strategy of cognitive offloading.

The Need for Cautious Skepticism

Of course, as a brief claim from social media, many details remain to be clarified:

• What exactly does "nearly solved" mean in terms of score? How much improvement is there compared to human baselines and previous models?
• What is the specific implementation of the coding harness? Is it a fully automated end-to-end pipeline, or does it involve significant manually designed prompts and tools?
• Is there a risk of overfitting to the benchmark? ARC's significance lies in testing generalization; if the solution is overly targeted, it may actually contradict the benchmark's original purpose.
• What are the computational costs? If extensive code generation-execution-correction loops are required, the inference-time compute may far exceed direct prediction — is this viable for practical deployment?

In the absence of a complete paper and reproducible data, we should treat this claim as a directional signal worth attention rather than a settled conclusion.

Conclusion

Though brief, this tweet encapsulates a key trend in current AI development: code is becoming a core lever for large models' journey toward general intelligence. From code pretraining to tool calling, to now using coding harnesses to tackle abstract reasoning benchmarks, we see a clear trajectory — teaching models to think like programmers may well be an important path toward stronger general capabilities.

Regardless of whether ARC-AGI-3 has truly been "nearly solved," this exploration itself reminds us: the boundaries of AI capability often depend not just on model scale but on how we guide models to think and solve problems. Code, as one of humanity's most precise tools for expressing thought, is becoming a key to unlocking AI's potential.