Gemini 2.5 Pro Coding Test: Crushes Claude at Rust Refactoring, Falls Flat on Frontend

Google's latest Gemini 2.5 Pro has been called "the best coding AI I've ever used" by many developers. A tech blogger put it through a multi-dimensional coding gauntlet—from pixel-art mini-games to Ultimate Tic-Tac-Toe, from landing page development to Rust code refactoring—with results that were both impressive and disappointing.

This article breaks down each test result to help you gauge exactly how capable Gemini 2.5 Pro's coding abilities really are, and where it's worth using versus where Claude still wins.

Game Development Tests: Pixel-Art Mini-Game & Ultimate Tic-Tac-Toe

Kitten Cannon Clone: Done in Three Prompts

The blogger first tackled a classic launcher game—a Kitten Cannon-style pixel-art game built with P5.js. P5.js is a JavaScript creative coding library built on the Processing language philosophy, designed for artists, designers, and programming beginners. It provides a clean API for Canvas-based 2D/3D rendering, animation, and interaction. It's extremely popular in the indie game development and creative coding communities because developers can quickly implement physics simulations, particle effects, and collision detection with minimal code. Kitten Cannon itself is a classic Flash web game from the mid-2000s where players launch a character and use various power-ups to extend flight distance for high scores—its core mechanics involve projectile motion simulation, collision detection, and score calculation.

The first generation produced 636 lines of code with a type error that prevented it from running. But by feeding the error message directly back to the model (without even providing additional context), Gemini 2.5 Pro successfully fixed the issue. After a total of three prompts, the game was up and running.

While it wasn't a "first-try success," the blogger raised a thought-provoking point: Five years ago we didn't even know AI could do this, and now it only takes one to three prompts to generate a complete pixel-art game. We're on a "hedonic treadmill" where our perception of technological progress keeps dulling, but objectively speaking, this AI code-generation capability remains stunning.

Ultimate Tic-Tac-Toe: First-Prompt Success

Even more surprising was the Ultimate Tic-Tac-Toe test. Ultimate Tic-Tac-Toe is a recursive upgrade of regular tic-tac-toe, proposed by mathematicians in the 2000s. The game requires nesting 9 small tic-tac-toe boards within a 3x3 macro grid, managing activation states, mouse input, win condition detection, and other complex logic. Its complexity far exceeds regular tic-tac-toe—regular tic-tac-toe has a state space of about 5,478 legal positions, while Ultimate Tic-Tac-Toe's state space grows exponentially, requiring simultaneous management of 81 cells, win/loss determination for 9 sub-boards, active board constraint rules, and global win conditions.

Gemini 2.5 Pro generated three Java Swing code files: MiniBoardPanel.java, GameBoardPanel.java, and UltimateTicTacToe.java. Java Swing is a GUI toolkit in the Java standard library, born in 1997. While it has been gradually superseded by frameworks like JavaFX in modern development, it still sees widespread use in education and enterprise legacy systems. The fact that AI can correctly generate Swing code demonstrates that its training data covers extensive knowledge of the classic Java ecosystem. After compiling and running according to its instructions, the game worked on the first try—all features functioned correctly, including board activation management and win determination. For a relatively "ancient" UI framework like Java Swing, this performance is quite impressive.

These two tests demonstrate that Gemini 2.5 Pro has strong game logic development capabilities, especially when handling complex state management and algorithm implementation.

Gemini 2.5 Pro Benchmark Results: The Numbers Speak

In formal benchmarks, Gemini 2.5 Pro's performance is equally impressive. Here's what each benchmark measures:

• Reasoning & Knowledge (Humanities Last Exam): An 18.8% score far exceeds O3 Mini's 14% and other models' sub-10% scores—a 34% improvement. HLE is an ultra-difficult test set co-designed by hundreds of humanities experts, covering philosophy, history, linguistics, and more. It tests AI's deep reasoning and interdisciplinary knowledge integration. Most models score below 10%, making 18.8% a significant breakthrough.
• Science (GPQA Diamond): 84 points, surpassing all other models' single-attempt scores. GPQA (Graduate-Level Google-Proof Q&A) Diamond contains science questions written by PhD-level experts that are difficult to answer even with Google searches, specifically measuring deep scientific reasoning capability.
• Math Ability: On par with O3 Mini, with both leaving other models far behind
• Code Editing (Aider Polyglot): Better than all competitors. Aider Polyglot is a multi-language code editing benchmark based on the open-source AI coding assistant Aider, testing model accuracy in code modifications across Python, JavaScript, Rust, and other languages.
• Agentic Coding (SWE-Bench Verified): Second only to Claude 3.7 Sonnet. SWE-Bench Verified simulates real software engineering tasks, requiring AI to autonomously locate code based on GitHub issue descriptions, write patches, and pass tests—currently one of the most authoritative benchmarks for measuring AI agentic coding capability.

One notable detail: while Claude 3.7 Sonnet slightly edges ahead on SWE-Bench, the blogger points out a practical issue: 3.7 Sonnet often does many things you didn't ask it to do, which actually led the blogger to revert to 3.5 Sonnet. This reminds us that benchmark scores don't equal real-world experience—when choosing AI coding tools, you also need to consider controllability and how well it follows instructions.

Training Data Freshness: Marketing vs. Reality

Gemini 2.5 Pro claims training data extending to March 2025, but when directly asked about the current React.js version, it answered 18.3.1 (information current as of May 2024). Only after enabling Google Search grounding did it correctly answer React 19.

This shows that there's a gap between the training data cutoff date and what the model actually knows. Developers should double-check framework versions and API changes. This phenomenon is actually common in large language models—the training data cutoff refers to the latest point of corpus collection, but the model's "memory" depth varies across knowledge domains. Information that appears frequently in training data gets encoded more firmly, while low-frequency or peripheral information may become fuzzy or even forgotten.

Frontend Development: Landing Pages & UI Cloning Are Clearly Weak

Landing Page Creation: Disappointing Results

The blogger tried having Gemini 2.5 Pro create a React + Tailwind CSS landing page from a design mockup. Tailwind CSS is a utility-first CSS framework that, unlike traditional component-based frameworks like Bootstrap, provides numerous atomic CSS classes (like p-4, text-center, bg-blue-500). Developers compose these classes directly in HTML to build styles, avoiding the tedium of writing custom CSS. Tailwind rapidly became a mainstream frontend choice after 2020 and is nearly standard in the React and Next.js ecosystem. However, Tailwind project initialization involves PostCSS plugin chains, purge configuration, and theme customization—a non-trivial challenge for AI.

While generation was fast (~70 seconds), the final result was quite poor.

Specific issues included:

• Typos in the Tailwind CSS installation commands—the fragmentation and rapid iteration of frontend toolchains makes it difficult for AI to consistently track the latest CLI commands and configuration formats
• Unable to directly download code for project import; each file had to be manually created
• Only included the background image, missing the logo and app icons
• The final result looked nothing like the design mockup—"this really isn't what a landing page should look like"

X (Twitter) UI Clone: Visually Okay but Functionally Empty

In the test to replicate X's desktop UI, Gemini 2.5 Pro used Google Search to view X's current interface and generated a single-HTML-file static page. The visual result was "not bad," but of course it couldn't implement any actual functionality.

The blogger's conclusion was direct: Frontend development is not Gemini 2.5 Pro's strong suit. If your primary work is frontend UI development, you'll probably still need to pair it with other AI tools.

Rust Code Refactoring: Gemini 2.5 Pro's Greatest Strength

If frontend is the weakness, Rust code refactoring is where Gemini 2.5 Pro truly shines. The blogger tested the task of "refactoring for loops into iterator methods" using the same Rust code across multiple AI models.

To understand why this task is so impressive, you need to know that Rust's Iterator pattern is a core expression of its zero-cost abstraction philosophy. Compared to traditional for loops, iterator chain calls (like .iter().filter().map().collect()) are not only more concise but also enable more aggressive compiler optimizations—the Rust compiler inlines and unrolls iterator chains, generating machine code that performs as well as or better than hand-written loops. This refactoring isn't simple syntax replacement; it requires deep understanding of Rust's ownership system, lifetimes, and borrowing rules.

Gemini 2.5 Pro's refactoring results were stunning:

• Completely eliminated all for loops, replacing them with idiomatic iterator methods (.iter(), .map(), etc.)
• Replaced Vec with slice, which better follows Rust best practices. A slice (&[T]) is a borrowed view of contiguous memory that avoids unnecessary ownership transfers, reduces memory allocation overhead, and improves function generality—a function accepting a slice can process data from Vec, arrays, and other contiguous memory containers
• Optimized logical structure, for example refactoring safety checks into more elegant trend detection (increasing or decreasing)
• Cleaned up surrounding code—not just mechanical replacement, but understanding code intent and performing holistic optimization

The blogger called this refactoring "better than Claude 3.5 and better than 3.7 too," saying the code was "really clean." This deep code comprehension and refactoring ability may be Gemini 2.5 Pro's most competitive feature in AI programming. For developers who regularly handle code optimization and refactoring tasks, this capability is extremely practical.

What Development Scenarios Is Gemini 2.5 Pro Best For?

Through these multi-dimensional tests, we can clearly see Gemini 2.5 Pro's strengths and weaknesses in programming:

Strong areas:

• Backend logic and algorithm implementation (Ultimate Tic-Tac-Toe succeeded on first try)
• Code refactoring and optimization (Rust refactoring far surpasses Claude)
• Game logic development (complex games completed with minimal prompts)
• Multi-language code editing (leads in Aider benchmark)

Areas needing improvement:

• Frontend UI development and landing page creation produce poor results
• Training data freshness doesn't match official claims
• Lacks deep IDE integration (not yet available in Windsurf and similar tools)

The IDE integration point deserves elaboration. Windsurf (from the former Codeium team) is a next-generation AI-native IDE, alongside Cursor, GitHub Copilot Workspace, and other cutting-edge AI-assisted programming products. The core value of these tools lies in deep integration of AI models with the development environment—beyond code completion, they can perceive the entire project's codebase context, file structure, dependencies, and terminal output, enabling cross-file editing, automatic debugging, and end-to-end feature development. Model capability and IDE integration are two independent dimensions of AI programming experience: even if the model itself is powerful, without IDE-level context injection and workflow orchestration, developers still need to manually copy-paste code and error messages, drastically reducing efficiency.

The blogger ultimately stated: Once Gemini 2.5 Pro is integrated into Windsurf, it might become the primary coding tool, because it's "really good and really cheap." This assessment likely echoes many developers' sentiments—we don't need an AI that's perfect in every way, but one that's powerful and reliable enough for core programming tasks.

For developers, the most pragmatic current strategy may be: Choose different AI coding tools based on task type. Use Gemini 2.5 Pro for code refactoring and backend logic; Claude might still be better for frontend development. The "do-it-all era" of AI programming tools hasn't arrived yet, but each iteration is narrowing the gap.

Key Takeaways

• Gemini 2.5 Pro delivers stunning code refactoring results, with Rust refactoring quality surpassing both Claude 3.5 and 3.7
• Outstanding backend logic development—Ultimate Tic-Tac-Toe Java Swing project generated successfully on the first try
• Frontend development is a clear weakness, with landing page and UI cloning results falling short
• Leads in multiple benchmarks, but training data freshness doesn't match the claimed March 2025 cutoff
• Excellent value for money—blogger believes it could become a primary coding tool once integrated with an IDE