Gemini 3.1 Pro In-Depth Comparison: 77.1% on ARC-AGI-2 — Is It Really the Best?

Google's latest Gemini 3.1 Pro has made quite a splash in the AI community — scoring 77.1% on the ARC-AGI-2 abstract reasoning test, hitting an ELO of 2887 in coding, and topping multiple core benchmarks. Does this mean the crown for the most powerful large language model has changed hands? Don't jump to conclusions just yet. This article provides a horizontal comparison across reasoning, coding, search, and more to help you see the full picture.

Three Core Upgrades: Reasoning, Coding, and Context

Gemini 3.1 Pro shows clear improvements over its predecessor in three key areas:

A major leap in reasoning ability. GPQA Diamond is a widely recognized high-difficulty reasoning benchmark — its questions draw from PhD-level knowledge in biology, chemistry, physics, and other fields, deliberately designed to resist search engine retrieval. Even domain experts average only about 65% accuracy. The Diamond subset contains the hardest questions, representing the ultimate test of a model's deep reasoning capabilities in academia. Gemini 3.1 Pro scored 94.3% on this test, directly surpassing OpenAI's o4 (OPAS 4.6) and GPT-5.2. This means its multi-step reasoning and cross-domain knowledge integration have significantly exceeded the average human expert level. That's an impressive score.

Comprehensive coding improvements. SWE-Bench score of 80.6%, TerminalBench at 68.5%, and a coding ELO rating of 2887 — placing it in the top tier across mainstream coding benchmarks.

Million-token context + multimodal input. Support for ultra-long context windows and multimodal input means practical usability has seen a qualitative improvement for real-world scenarios like processing large codebases and long document analysis.

ARC-AGI-2 Abstract Reasoning: A Dominant Lead

ARC-AGI-2 (Abstraction and Reasoning Corpus for Artificial General Intelligence) was designed under the leadership of AI safety researcher François Chollet as the second generation of the ARC test series. Its core design philosophy is that true intelligence should not rely on memorization or statistical patterns, but should possess the ability to induce abstract rules from a small number of examples. Test problems typically present colored grid patterns, requiring the model to identify hidden transformation rules and infer outputs — tasks that are intuitively straightforward for humans but extremely challenging for language models that depend on large-scale pretraining data. ARC-AGI-2 is significantly more difficult than the first generation, incorporating more complex spatial reasoning and multi-step logic chains. The research community considers it an important indicator for measuring a model's "genuine reasoning ability" rather than "memory retrieval ability."

On this test, the gap between models is clear:

Model	ARC-AGI-2 Score
Gemini 3.1 Pro	77.1%
OpenAI o4	68.8%
GPT-5.2	52.9%

Gemini 3.1 Pro leads by a wide margin at 77.1%, more than 8 percentage points above o4 and a full 24 percentage points ahead of GPT-5.2. In the dimension of pattern recognition and abstract logical reasoning, Google's new model has truly achieved a qualitative breakthrough.

Another noteworthy highlight is search capability. BrowseComp is a web browsing and information retrieval benchmark released by OpenAI that tests a model's comprehensive ability to perform multi-hop searches, information synthesis, and fact-checking in open web environments. It evaluates not just single-query retrieval accuracy, but also cross-page reasoning and information denoising. Gemini 3.1 Pro scored 85.9% on this test — another dominant lead. This is inseparable from Google's 25+ years of experience in web indexing, PageRank algorithm iterations, and real-time search infrastructure. These engineering capabilities have been systematically integrated into Gemini's training data quality and retrieval-augmented architecture, forming a competitive moat that other companies cannot easily replicate in the short term.

Not a Complete Domination: Weaknesses Exist Too

However, labeling Gemini 3.1 Pro as "the strongest across all dimensions" would be an overstatement.

Slightly behind on SWE-Bench Pro. It's worth noting that SWE-Bench and SWE-Bench Pro are not the same difficulty level: the standard SWE-Bench tests relatively independent single-file fix tasks, while the Pro version introduces more complex multi-file modifications, longer code dependency chains, and stricter test verification standards — much closer to the complex tasks real engineers face daily. On the harder SWE-Bench Pro test, Gemini 3.1 Pro scored 54.2%, while GPT-5.2 achieved 55.6%. The gap is small, but it shows that OpenAI's model still holds an edge on complex engineering-level coding tasks, revealing a capability gap in Gemini 3.1 Pro between "simple coding tasks" and "engineering-level complex tasks."

Uncertainty from Preview status. Gemini 3.1 Pro is still in Preview stage, and the official release may undergo further adjustments and optimizations. Current benchmark scores are subject to change and shouldn't be over-interpreted.

Gaps remain in specialized domain tasks. On the GDPo expert task evaluation, Gemini 3.1 Pro scored 1317, below Anthropic Claude's equivalent level. This means Google's model still has room for improvement in certain vertical scenarios requiring deep specialized knowledge.

Ecosystem Strategy: The Anti-Gravity Agent Platform

Beyond the model upgrades themselves, Google also simultaneously launched the Anti-Gravity agent platform. AI Agents are one of the core paradigms for large model deployment today — referring to AI systems capable of autonomously planning tasks, invoking tools, executing multi-step operations, and dynamically adjusting strategies based on environmental feedback, distinct from traditional "Q&A-style" interactions.