Google Jules Hands-On Review: Testing the Real Capabilities and Limitations of This AI Coding Agent with a Java Project

Google recently launched Jules, a free AI coding agent that the company calls "the future direction of coding agents." It's not a traditional code completion tool — it's an intelligent agent that can autonomously clone repositories, understand code intent, and independently complete development tasks. Sounds enticing, but how does it actually perform in practice? Lin Xi, an engineer from the LexiLens development team, conducted an in-depth test using a real Java backend project, and the results are worth a close look for any developer following AI coding tools.

What Is Jules? How Does It Differ from Copilot and Other AI Coding Assistants?

In its official blog, Google states that agentic development is moving from prototype to product and is rapidly becoming a core part of how software is built. Jules is powered by Google's Gemini 2.5 Pro model, paired with a cloud-based virtual machine system, enabling it to handle complex multi-file changes and concurrent tasks.

Gemini 2.5 Pro and Cloud VM Architecture: Gemini 2.5 Pro is a multimodal large language model released by Google DeepMind in 2025 that demonstrates outstanding performance in code comprehension and generation benchmarks, particularly surpassing multiple competing models on programming evaluation sets like HumanEval and SWE-bench. Jules combines it with a cloud sandbox VM, allowing the model not only to "read" code but also to "run" code in an isolated environment — a key architectural difference from pure language model completion tools. This "read-think-execute" closed-loop design theoretically reduces hallucinations, but in actual testing, it still called non-existent methods, indicating that an information gap persists between the cloud execution environment and the model's code understanding layer.

The Technical Paradigm of AI Coding Agents: AI coding agents represent the next generation of programming assistance beyond code completion (the Copilot paradigm). Their core architecture typically includes four modules: a Task Planner, a Tool Use layer (file read/write, terminal execution, Git operations), Context Window Management, and a Feedback Loop. Jules' "plan first, execute later" mode is an engineering implementation of the ReAct (Reasoning + Acting) framework, proposed by Google researchers in 2022 to have models explicitly reason before acting, thereby improving interpretability and controllability. Compared to similar products like Devin and OpenHands, Jules differentiates itself through deep native integration with the GitHub ecosystem rather than functioning as a standalone IDE plugin.

Compared to tools like GitHub Copilot and Cursor, Jules works more like a "remote developer":

• Autonomous repository cloning: Pulls your code directly from GitHub
• Independent analysis and planning: Creates an execution plan first, then proceeds after user confirmation
• Cloud VM execution: Tasks continue running even after you close the browser
• Direct branch and PR creation: Can submit code changes directly upon completion

Jules is currently free to use with a daily quota of 60 tasks and deep GitHub integration. Anyone with a Google account who meets access requirements can start using it right away.

Hands-On Testing: A Complete Record from Project Analysis to Code Refactoring

Step 1: Having Jules Comprehensively Analyze the Java Project

The test used a real Java backend project. The first task given to Jules was: "Please comprehensively analyze this project from multiple perspectives including performance, code quality, and coding standards."

Jules first produced an analysis plan covering project structure review, coding standards audit, performance evaluation, documentation quality, and test coverage. After user confirmation, it began executing the analysis autonomously.

A small hiccup occurred during the process — Jules mistakenly identified the project name as the sponsor name "Lumino," showing that deviations can occur even in the initial understanding phase.

Another detail worth noting: Jules has a 5-minute time limit per task. During this analysis, it couldn't complete all the work within the allotted time and needed user feedback to continue.

The final analysis report was of decent quality. It correctly identified the project's main strengths (strict naming conventions, comprehensive JavaDoc documentation, appropriate use of design patterns) and performance advantages (high-performance caching, asynchronous concurrent processing), while also pointing out a real issue: serialization and deserialization in distributed components (gRPC, Redis) is a potential performance bottleneck. This finding perfectly aligned with issues the developer already knew about.

Step 2: Using Protocol Buffers to Solve the Serialization Bottleneck

Based on the serialization issue identified in the analysis report, the tester gave Jules a second task: "Do you have any solutions? Can you implement one for me?"

Jules researched multiple serialization libraries (Kryo, Jackson, Protocol Buffers) and ultimately recommended Protocol Buffers as the primary solution, with Kryo as a fallback.

Why Protocol Buffers Is a Reasonable Choice: Protocol Buffers (Protobuf) is Google's open-source binary serialization protocol that reduces payload size by approximately 60-80% compared to JSON and parses 3-10x faster. It's the default serialization format for gRPC communication. In Java backend projects, Redis caching typically uses JSON or Java native serialization, but both have clear performance bottlenecks under high concurrency — Java native serialization has poor security and large payload sizes, while JSON has high CPU overhead. Jules' recommendation of Protobuf as the Redis serialization solution is a sound technical judgment, but its failure to include Maven/Gradle dependency declarations — a fundamental step — in the implementation exposes the capability gap between "knowing what to use" and "knowing how to fully configure it" — a typical shortcoming of current coding agents.

After receiving confirmation, Jules began refactoring the code, specifically:

• Refactoring service and task modules to use Protocol Buffers for Redis serialization
• Adding new test classes for unit testing
• Creating a separate branch and committing changes (800 lines of code modified in total)

The commit messages were fairly clear, and the branch naming was reasonably standard. From a workflow perspective, Jules did demonstrate what an "autonomous agent" should look like.

Step 3: Code Quality Verification — Problems Emerge in Clusters

However, when the tester opened Jules' submitted code in the IDE, problems surfaced one after another:

1. Missing dependencies: Jules used the Protocol Buffers library but didn't add the corresponding dependency to the project build file, resulting in numerous compilation errors
2. AI hallucination: Jules called a method that doesn't exist at all, showing it didn't truly understand the project's complete API
3. Inconsistent coding style: The generated code was noticeably different from the project's existing coding conventions — for example, it didn't use the project's existing logging components
4. Missing JavaDoc: The original project had comprehensive JavaDoc comments, but Jules' new code completely failed to follow this convention
5. Incorrect test implementation: The initial test methods were written incorrectly and required manual guidance to correct

The Special Danger of AI Hallucinations in Code Generation: The "hallucination" phenomenon in large language models is far more dangerous in code generation scenarios than in text generation. Text hallucinations may produce incorrect information, but code hallucinations directly cause compilation failures, runtime crashes, or even security vulnerabilities.