Gemini 3.5 Pro Hands-On: No More Lazy Code Generation, Complete Projects in a Single Output

The Persistent Plague of Lazy AI Coding: A Universal Developer Pain Point

If you've been using large language models for coding assistance for any length of time, you're undoubtedly familiar with the so-called "laziness" phenomenon. This term describes a model's behavior of deliberately omitting implementation details to save on output — common manifestations include placeholder comments like // TODO: implement this, shortcuts like // ... rest of the code stays the same, or simply providing function signatures without filling in the logic.

From a technical perspective, this phenomenon is closely tied to the autoregressive generation mechanism of the Transformer architecture. When a model generates output token by token, each step requires attention computation over all previously generated tokens, and the computational cost grows significantly as outputs get longer. Specifically, the Self-Attention mechanism in the Transformer architecture needs to compute attention weights between each new token and all preceding tokens, with a computational complexity of O(n²), where n is the sequence length. This means that when the model generates the 1,000th token, it needs to review information from the previous 999 tokens to decide what to output next. For inference processes optimized with KV Cache, while redundant computation is avoided, memory usage still grows linearly with sequence length. This architectural characteristic causes the marginal cost of generating long sequence outputs to continuously increase, creating implicit pressure on output length under resource-constrained conditions.

Additionally, the model's training data contains vast amounts of Stack Overflow answers, technical blog posts, and code snippets — content that inherently tends to showcase core logic rather than complete implementations. When the model learns this distributional pattern, it naturally gravitates toward generating "tutorial-style" code snippets rather than production-grade complete implementations.

For developers who want to get fully runnable code in a single pass, this lazy behavior drastically reduces efficiency, often requiring multiple rounds of follow-up prompting to piece together a complete implementation. According to a recent report from a Reddit community user, the upcoming Gemini 3.5 Pro may bring significant improvement on this pain point.

Arena Battle Testing: Gemini 3.5 Pro's Code Output Capability Dramatically Improved

The user reported that Gemini 3.5 Pro is currently being tested in Arena (an LMArena-type competitive platform) Battle Mode in the form of multiple checkpoints.

LMArena (formerly LMSYS Chatbot Arena) is a large model evaluation platform created by a UC Berkeley research team that uses an ELO rating system to rank models. The ELO rating system was originally designed by physicist Arpad Elo for chess, with the core idea of dynamically updating player ratings based on match outcomes — defeating a higher-rated player earns more points, while losing to a lower-rated player costs more. LMArena applies this mechanism to AI model evaluation: each time a user votes for the better response, it's equivalent to the result of a "match," and the system updates both models' ELO scores accordingly. After tens of thousands of user votes, model ELO scores stabilize, forming reliable rankings. As of 2024, LMArena has accumulated over 1 million human votes, and its rankings correlate highly with multiple professional benchmarks, making it an industry-recognized reference standard for model capability.

The core design philosophy of its Battle Mode draws from competitive rating systems like chess: after a user submits a prompt, the system randomly selects two models to generate responses, and the user chooses the better result without knowing the models' identities. This double-blind evaluation mechanism effectively avoids brand bias and is considered the evaluation method closest to real user preferences in the industry. Companies like Google and OpenAI frequently deploy new models to Arena for A/B testing before official release.

In large model development, a checkpoint refers to a snapshot of model weights saved at a certain point during training. A model typically goes through dozens or even hundreds of checkpoints from pre-training to final release, with each version potentially performing differently across various capability dimensions. Placing multiple checkpoints into Arena testing simultaneously is an efficient human evaluation strategy that helps teams quickly identify which training stage's model performs best in real user interactions, thereby determining which version to ultimately release or whether further tuning is needed.

According to the user's observations, compared to its predecessor Gemini 3.1 Pro, the new version shows significant improvement in code generation completeness:

"It can generate over 20 different files in a single pass, with hundreds of lines of code each — more than any AI model I've previously tested."

If this description is accurate, it means Gemini 3.5 Pro can output a codebase with a complete project structure in a single response, rather than scattered fragments. Generating over 20 files with hundreds of lines each could involve continuous generation of tens of thousands of tokens, posing multiple technical challenges for the model. First is the attention dilution problem: as output grows longer, the model's attention weights on early context become diluted, potentially causing interfaces in later files to be inconsistent with earlier ones. Second is repetition degeneration: the model can fall into repetitive patterns during long sequence generation. Technical approaches to solving these problems include: long-context attention mechanisms like Ring Attention or Sliding Window Attention, improved sampling strategies (such as presence penalty and frequency penalty), and specifically including long documents and complete codebases in training data to help the model learn long-range dependencies. Google's Gemini series is known for supporting ultra-long contexts (up to 2 million input tokens), and this architectural advantage may also contribute to its long output capability.

For real-world engineering scenarios requiring scaffolding and multi-module coordination, this is undoubtedly a significant advantage.

Caution Still Warranted

It's important to emphasize that the above conclusions currently come only from a single community user's personal observations, not from an official Google release or third-party benchmark testing. The "multiple checkpoint" versions in the arena itself indicate the model is still in the tuning phase, and the final release version's performance may differ from current test snapshots. Furthermore, "generating more code" and "generating higher quality code" are not the same thing — high output volume can also come with redundancy, hallucinations, or structural bloat. Therefore, this leak is better treated as a signal worth watching rather than an established fact.

Why Do AI Models Get Lazy with Code Generation? Three Root Causes

Model laziness is not accidental but the combined result of training objectives and inference mechanisms.

Cause One: Implicit constraints on output length. Large models are exposed to vast amounts of real code and documentation with omission conventions during training — human programmers themselves habitually use comments instead of repetitive code, and models learn this pattern of "reasonable omission." Notably, code examples in training data typically come from tutorials and documentation, which naturally present content in snippet form rather than as complete production codebases. After training on massive amounts of such data, models develop an output prior of "providing the key parts is sufficient."

Cause Two: Preference bias during alignment. During the RLHF (Reinforcement Learning from Human Feedback) phase, concise, focused answers tend to receive higher scores, which may inadvertently encourage a "just touch on it" output style. The RLHF workflow involves: first collecting human annotators' preference rankings of multiple model outputs, then training a reward model to predict human preferences, and finally using reinforcement learning algorithms like PPO (Proximal Policy Optimization) to maximize the language model's reward model scores. PPO was proposed by OpenAI in 2017 and became the mainstream choice for RLHF due to its training stability and low hyperparameter sensitivity. In the RLHF workflow, PPO's role is to maximize the reward model's scores while maintaining the language model's original language capabilities (by constraining the KL divergence from the base model). The entire process requires running four models simultaneously: the current policy model, a reference model (for KL constraints), a reward model, and a value function model, making RLHF training extremely computationally expensive. Recent alternative methods like DPO (Direct Preference Optimization) attempt to simplify this workflow, but PPO still holds advantages in complex preference alignment tasks.

The problem is that human annotators evaluating coding outputs tend to prefer structurally clear, focused answers over lengthy complete code — because within limited evaluation time, concise answers are more easily judged as "high quality." This annotation bias is transmitted to the language model through the reward model, creating an incentive structure for "laziness."

Cause Three: The tradeoff between context and compute. Generating extremely long code consumes substantial tokens and inference resources, and models tend to converge their output under internal "budget" constraints. From the service provider's perspective, every additional generated token means extra GPU computation and latency costs. Some models may have soft output length caps imposed at the system level, or length penalties applied during training to control output scale — measures that control costs but also exacerbate lazy tendencies.

For these reasons, if Gemini 3.5 Pro can reliably output complete multi-file projects, it strongly suggests that Google has made targeted optimizations in training data composition, alignment reward design, or long-text generation capabilities. Specifically, possible technical approaches include: significantly increasing the proportion of complete open-source projects in training data, specifically setting scoring dimensions for code completeness in reward model training, and expanding the model's maximum output token limit while optimizing the stability of long sequence generation.

What Does a Non-Lazy Gemini 3.5 Pro Mean for Developers

For engineers who deeply integrate AI into their daily development workflow, a "non-lazy" model delivers tangible efficiency gains:

• Fewer round-trip conversations: Getting a complete implementation in one pass eliminates the communication overhead of repeatedly prompting "please complete the omitted parts." In real-world development, each follow-up round not only consumes time but may also cause the model to "forget" earlier constraints due to limited context windows, producing inconsistent code.
• Better suited for Agent scenarios: In automated programming Agents, whether a model can reliably output complete code directly determines whether the task chain can execute smoothly. AI programming Agents (such as Devin, OpenHands, Claude Code, etc.) operate in a fundamentally different way from traditional chat-style coding assistants — Agents typically adopt architectural patterns like ReAct (Reasoning + Acting) or Plan-and-Execute, needing to autonomously plan tasks, generate code, execute code, observe results, and iterate on corrections, forming an automated task chain throughout the process. An Agent first receives user requirements, breaks them down into subtasks through reasoning (such as creating files, writing functions, running tests), then executes each subtask sequentially while observing environment feedback (such as terminal output, test results), and decides the next action based on feedback. This cycle continues until the task is complete or the maximum iteration count is reached. If the underlying model outputs incomplete code at any step (such as containing TODO placeholders), the Agent's execution step will directly error out, causing the entire task chain to break and requiring additional repair iterations — in severe cases, even falling into infinite loops. Therefore, the completeness and executability of model output is a fundamental prerequisite for reliable Agent operation.
• Enhanced project scaffolding capability: Multi-file coordinated generation makes requests like "build an application from scratch" closer to out-of-the-box ready. This means developers can obtain a complete project skeleton including routing configuration, data models, API endpoints, frontend components, and full directory structure through a single prompt, dramatically shortening the time from concept to runnable prototype.

However, developers should still maintain critical usage. More code doesn't mean you can skip review — especially for aspects involving security, performance, and business logic correctness, human oversight remains indispensable. Large volumes of generated code may harbor subtle logic errors, insecure dependency calls, or design decisions incompatible with a project's existing architecture — problems that are often harder to detect than missing code.

Conclusion: Worth Looking Forward To, But Rationality Still Required

The claim that Gemini 3.5 Pro is "no longer lazy" is still in the early stages of community testing, and both its authenticity and stability await official release and broader benchmark validation. But it touches on a core experience issue that all AI coding users care about. If Google has indeed achieved a breakthrough in this dimension, the Gemini series' competitiveness in the AI-assisted coding space will be further strengthened — especially in direct comparisons with competitors like Claude 3.5 Sonnet and GPT-4o, where code output completeness is becoming a key dimension of differentiation.

Let's wait and see how the official release performs, while also reminding all developers: until official data is released, any leak from a single source deserves an extra measure of rational scrutiny.