Gemini 3.1 Pro Coding Real-World Test: #1 in Benchmarks, #3 in Practice — An Honest Comparison with Claude and GPT

Introduction: Does #1 in Benchmarks Mean Best at Coding?

Google's recently released Gemini 3.1 Pro delivers impressive results across various benchmarks, with coding ability scores leading in nearly every dimension. But benchmarks are just benchmarks — what developers truly care about is real-world performance in actual programming scenarios.

The AI coding benchmark ecosystem typically includes standard test suites like HumanEval, MBPP, and SWE-bench. HumanEval, proposed by OpenAI, contains 164 Python programming problems that primarily test function-level code generation. SWE-bench comes closer to real-world scenarios by requiring models to solve actual GitHub issues. However, these test suites share common limitations: problems are relatively isolated, contexts are simple, and they don't involve multi-file collaboration or cross-project comprehension. In real development scenarios, developers face thousands of lines of existing codebases, complex dependency relationships, and ambiguous requirement descriptions — dimensions that are virtually absent from standard benchmarks.

This article is based on a developer's hands-on comparison, using identical prompts and real projects to evaluate Gemini 3.1 Pro against Claude Opus 4.6 and GPT 5.3 Codex. Two coding challenges cover multiple dimensions including cross-project migration, architecture adaptation, full-stack development, and streaming output. The final results are surprising.

Test Design: The Thinking Behind Two Real Programming Challenges

To ensure fairness and reproducibility, the tests used prompts identical to previous evaluations without any modifications:

Task 1: Cross-Project Authentication System Migration

• Migrate a complete user authentication system (Email, Google, GitHub login) into an image generation Agent project
• Simultaneously create a landing page for the project
• Skills tested: code comprehension, architecture adaptation, engineering standards

Task 2: CLI Agent to Web UI Conversion

• Transform a terminal CLI Agent tool into a web chat interface
• Preserve Syncing, tool calls, streaming output, and other features
• Skills tested: full-stack development, streaming output, UI/UX interaction

Task 1 Results: Gemini 3.1 Pro Stumbles on Complex Project Migration

Gemini's Five-Round Fix Process

Gemini 3.1 Pro performed poorly on the first task, taking five rounds of conversation to barely complete it, with the final result still showing obvious issues:

1. First round: Code generation completed, but the project wouldn't start
2. After continued fixes: Project could run, but page navigation had problems
3. Core bug: After successful login authentication, it failed to correctly redirect to the image generation Agent's main page
4. Critical comprehension error: The landing page completely copied content from the reference project instead of generating content based on the current project

This error exposed Gemini 3.1 Pro's weakness in prompt comprehension quality. Prompt comprehension quality is one of the core metrics that differentiates AI coding assistants' capability levels, involving the model's instruction following ability and contextual reasoning capacity. High-quality prompt comprehension requires the model not only to parse explicit instructions but also to infer implicit constraints — for example, the instruction "migrate the authentication system to the target project" implicitly constrains that "landing page content should be based on the target project, not the reference project." Research shows that models processing complex prompts with multiple entity references are prone to Entity Confusion — incorrectly mapping similar concepts from different contexts to the same object — which is particularly fatal in code generation scenarios.

Cross-project code migration is itself a recognized high-difficulty task in software engineering. The model needs to simultaneously understand the architecture of both source and target projects, distinguishing which parts are reusable generic logic and which are project-specific implementations requiring adaptation. Authentication systems are typically deeply coupled with database schemas, session management, route guards, environment variables, and more — any misjudgment in these components leads to runtime errors. Gemini's "copying reference project content" problem is essentially "attention drift" during long-context reasoning — an inability to precisely distinguish boundaries between two projects, academically known as "Lost in the Middle."