Gemini 3.5 Checkpoint Leaked: Intermediate Version Reportedly Outperforms Opus 5 in Testing

A Model Checkpoint Sparks Community Buzz

Recently, discussions surfaced on Reddit about an intermediate checkpoint of Google's Gemini 3.5. According to the leak, this yet-to-be-released model version outperformed Anthropic's Claude Opus 5 (in its highest reasoning mode, max thinking) on a certain test. The news quickly ignited heated debate across the AI community — although the source is limited and details remain incomplete, it reflects the increasingly fierce competition among top-tier LLM providers.

It's worth noting that the original information this article draws on is quite brief and comes from community leaks, with no official confirmation from Google. The following analysis is more of an extrapolation based on available information than a set of definitive conclusions.

What Is a Model Checkpoint?

A Weight Snapshot During Training

In large model training, a checkpoint refers to a snapshot of the model's weights saved at a particular point during the training process. It's not the final release version but rather an intermediate state during R&D. Teams typically save multiple checkpoints for evaluating training progress, conducting A/B tests, or as rollback backups.

To understand the significance of checkpoints, you first need to grasp the basics of large model training. Modern large language models (LLMs) are essentially neural networks containing billions or even trillions of parameters. During training, the model repeatedly processes massive amounts of text data, using Gradient Descent algorithms to continuously adjust these parameters (also called "weights") so that the model's output progressively approaches the desired result. This process can last weeks or even months, consuming the compute power of thousands of GPUs. Because training takes so long and is extremely expensive (a single complete training run can cost tens of millions or even hundreds of millions of dollars), teams periodically save all current parameter values during training — that's a checkpoint. If a hardware failure, loss function divergence, or other unexpected issue occurs during training, the team can recover from the most recent checkpoint instead of starting from scratch. Additionally, checkpoints are used for periodic evaluation: researchers run benchmark tests on checkpoints from different stages, plotting "capability growth curves" to determine whether training is converging in the right direction.

The "Gemini 3.5 checkpoint" mentioned in the leak is most likely a version used internally by Google for testing or comparative evaluation. Performance data from such internal versions often leaks before official release, becoming an important window for outsiders to glimpse a company's technical progress. In industry practice, a model typically goes through several critical steps between checkpoint and official release: instruction tuning, alignment training such as Reinforcement Learning from Human Feedback (RLHF) or Direct Preference Optimization (DPO), as well as safety evaluations and red-team testing. These steps may further improve the model's practical performance, or may cause some dimensions to regress due to safety constraints.

Why Checkpoint Performance Matters

If an intermediate version that hasn't completed full training and tuning can already beat a competitor's flagship model on a specific test, it suggests at least two things:

• A breakthrough in training methodology: Google may have made substantive progress in its training strategy for the Gemini series. This could involve innovations across multiple dimensions — such as more efficient pretraining data mixture strategies, improved attention mechanism architectures, better learning rate scheduling, or further optimization of the Mixture of Experts (MoE) architecture. Google DeepMind has extensively adopted MoE architecture in the Gemini series, which allows different "expert" sub-networks to handle different types of input, dramatically increasing the model's total parameter count and expressiveness while maintaining computational efficiency.
• A higher capability ceiling: The final release version, after complete training and alignment, could push the performance ceiling even further.

Of course, results from a single test cannot represent a model's overall capabilities. Different benchmarks evaluate very different dimensions, and winning one test doesn't necessarily mean comprehensive superiority.

What Does Beating Opus 5 Max Thinking Mean?

A Head-to-Head in Reasoning Mode

Claude Opus 5's "max thinking" refers to the model's performance at maximum reasoning compute — allowing the model to engage in longer, deeper chain-of-thought reasoning. This mode typically represents the ceiling of a model's capabilities on complex reasoning tasks.

This involves an important paradigm shift in the LLM field in recent years: Test-time Compute Scaling. Traditionally, the primary way to improve model capability was to invest more computational resources during the training phase — more data, more parameters, longer training times. But starting in 2024, the industry discovered that giving models more "thinking time" during inference (i.e., when generating responses) can also significantly boost performance. The core of this approach is Chain-of-Thought Reasoning: instead of directly outputting an answer, the model first generates a step-by-step reasoning process — decomposing the problem, listing known conditions, working through calculations step by step, verifying results, and only then providing the answer. OpenAI's o1/o3 series, Anthropic's Claude Extended Thinking mode, and Google's Gemini thinking mode are all products of this paradigm. "Max thinking" means maxing out the compute budget for this reasoning process, allowing the model to generate extremely long internal reasoning chains (potentially tens of thousands of tokens), repeatedly verifying and correcting on the most complex problems. Performance in this mode is typically regarded as the true upper bound of a model's reasoning ability.

If an intermediate checkpoint of Gemini 3.5 can surpass Opus 5 at its highest reasoning state on a given test, the comparison carries particular weight: it means Google's model can come out on top in specific tasks even when the competitor is going all out.

The Need for Caution

That said, we must maintain rational judgment about such leaks:

1. Insufficient sample size: Results from "one test" lack statistical persuasiveness
2. Opaque testing conditions: The specific content, evaluation methodology, and difficulty level are all undisclosed, making it impossible to rule out cherry-picking
3. Risk of information distortion: Community leaks often contain exaggeration or misinterpretation

Truly valuable evaluation should be based on open, reproducible, multi-dimensional benchmarks such as MMLU, GPQA, SWE-bench, AIME, and other standardized assessments.

These benchmarks each have different focuses, together forming a multi-dimensional framework for evaluating LLM capabilities:

• MMLU (Massive Multitask Language Understanding): A large-scale multitask knowledge comprehension test covering 57 subject areas (from astronomy to law, mathematics to medicine), containing approximately 14,000 multiple-choice questions. It's a classic benchmark for measuring a model's breadth of knowledge and foundational understanding.
• GPQA (Graduate-Level Google-Proof Q&A): A high-difficulty Q&A test with questions created by PhD-level experts across various fields, carefully designed to ensure answers can't be easily found even with search engines. This benchmark specifically tests a model's deep reasoning ability in physics, chemistry, biology, and other disciplines, and is considered one of the most discriminating academic reasoning evaluations available.
• SWE-bench: A software engineering task benchmark drawn from real GitHub projects, requiring models to read code repositories, understand issue descriptions, and generate code patches that solve actual bugs or implement feature requirements. It tests not just code generation ability, but a model's understanding of complex engineering contexts and problem-solving capability.
• AIME (American Invitational Mathematics Examination): Questions from the American math competition, at the high school math olympiad level, requiring creative mathematical reasoning and multi-step problem solving. It's an important benchmark for testing a model's mathematical reasoning ability.

Only when a model performs well across these tests spanning knowledge, reasoning, coding, mathematics, and other dimensions can it be considered to have a genuine comprehensive advantage.

A Microcosm of the Top-Tier LLM Competition

Continuous Escalation Across Three Major Camps

Behind this news lies the ongoing arms race among OpenAI, Google, and Anthropic in frontier models. Each version iteration brings comprehensive upgrades in parameter scale, training data, reasoning capability, and multimodal performance.

• Google Gemini Series: Evolving from 2.0 to 2.5 and beyond, it has established unique advantages in multimodality and long context. Google DeepMind's core technical approach is "Natively Multimodal" — unlike earlier approaches that bolted vision and language modules together, Gemini is designed from the architectural ground up to simultaneously process text, images, audio, and video. Furthermore, Gemini has consistently led in context window length, with Gemini 2.5 Pro supporting a 1-million-token context (roughly equivalent to several complete books), providing a significant advantage in tasks like long document analysis and large-scale code repository comprehension. Google also holds a unique infrastructure advantage — its self-developed TPU (Tensor Processing Unit) chips and the world's largest-scale compute clusters give it structural competitiveness in training efficiency and cost control.
• Anthropic Claude Series: Known for strong reasoning and high reliability, the Opus series has long been regarded as the benchmark for complex reasoning tasks. Anthropic was founded by former core OpenAI researchers, and the company's technical philosophy centers on AI safety. It pioneered the "Constitutional AI" alignment method — achieving a balance between safety and capability by having AI critique and correct itself according to a set of explicit principles. The Claude series excels in long-text comprehension, code analysis, instruction following, and hallucination reduction, making it highly popular among enterprise users and developers. Its reasoning mode (Extended Thinking) employs a fine-grained "thinking budget" mechanism, allowing users to flexibly allocate reasoning compute based on task complexity.
• OpenAI GPT/o Series: As the pioneer of LLM commercialization, OpenAI has both GPT-4o as a flagship model for general-purpose scenarios and the o1/o3 series focused on deep reasoning. OpenAI still leads in productization and ecosystem development — ChatGPT has the largest end-user base, and its API ecosystem and plugin system are the most mature. On the technical front, OpenAI was the first to apply reinforcement learning at scale to reasoning task training, creating the new category of "reasoning models."

Competition among the three camps is no longer limited to model performance itself but extends across the entire value chain — infrastructure, developer ecosystems, enterprise services, and application scenarios.

The fact that they're competing at the checkpoint stage shows that competition has drilled down to every link in the R&D process.

Naming Cadence Reflects Iteration Speed

Also worth noting is the acceleration in version naming. From Gemini 3.5 to Opus 5, version numbers across companies are climbing rapidly, reflecting a significant shortening of model iteration cycles.

Looking back at recent iteration cadence, this acceleration trend is very clear: when GPT-4 was released in early 2023, there was roughly a four-month gap from GPT-3.5, while GPT-3.5 was more than two years after GPT-3. By 2024-2025, top-tier companies are releasing a major version update or entirely new model series roughly every 2-3 months. Anthropic advanced from the Claude 3 series to the Claude 5 series in less than a year and a half; Google's Gemini has rapidly progressed from 1.0 to 3.5. Multiple factors drive this acceleration: expanding compute infrastructure shortens training cycles, automated evaluation and alignment techniques reduce the labor cost of model tuning, and intense market competition forces every player to convert research breakthroughs into products as quickly as possible.

This high-frequency iteration drives rapid capability improvement on one hand, but on the other, it creates "version fatigue" for users and developers — just as you've adapted to one model's characteristics and limitations, the next version arrives. For enterprises building applications on LLMs, this means establishing more flexible model-switching mechanisms and abstraction layers to avoid over-reliance on any single model version. The rise of development frameworks (such as LangChain, LlamaIndex) and model routing services is a direct response to this pain point — they help developers switch seamlessly between different models, reducing migration costs caused by model iterations.

Takeaways for Developers and Users

Don't Let a Single Data Point Drive Your Technology Choices

For developers building applications on large models, technology selection decisions should not be based on scattered leaks or single test scores. A more pragmatic approach is to:

• Build a custom evaluation set tailored to your actual business scenarios
• Conduct side-by-side comparison tests across candidate models
• Holistically consider multiple factors including performance, cost, latency, and API stability

After all, strengths and weaknesses on general benchmarks don't necessarily translate to suitability for specific tasks. A model that scores highest on MMLU might underperform a competitor specifically optimized for your particular domain (such as legal contract review or medical imaging report generation). Moreover, model selection in production environments requires balancing inference costs (per-token pricing can vary by more than 10x), response latency (real-time conversation scenarios are extremely sensitive to latency), context window size, degree of multimodal support, and non-performance factors like vendor service reliability and data privacy guarantees.

Focus on Official Releases and Technical Reports

What truly deserves your attention is the technical report and official benchmark data published when Google formally releases Gemini 3.5. Only then can you obtain a credible, comprehensive performance profile. Until then, community leaks can serve as trend references but should not be used as a basis for decision-making.

It's worth mentioning that technical reports from LLM providers themselves also require careful interpretation. Companies typically choose the most favorable benchmarks and evaluation settings when showcasing their results, and self-reported results lack independent third-party verification. Therefore, it's equally important to follow independent evaluation organizations and community reproduction results (such as the LMSYS Chatbot Arena crowdsourced rankings, Scale AI's SEAL leaderboard, etc.). These third-party evaluations, through blind comparisons and randomized experimental designs, can provide a more objective picture of model capabilities.

Conclusion

Whether or not the rumor that a Gemini 3.5 checkpoint "beat Opus 5" turns out to be true, it offers us a snapshot of the frontier of LLM competition. It reminds us that the technology race among top-tier providers has entered a white-hot phase, where breakthroughs can come from any training iteration.

But as rational technology observers, we should maintain caution amid anticipation — the real answers still await official releases and public evaluations. In an era of rapid AI evolution, the only certainty is this: the capability improvements driven by competition will ultimately benefit every user.