Antigravity Hands-On Review: Model Failures and Quota Anxiety — A Developer's Real Talk

A Student Developer's Real Struggle

Recently, a student developer posted an in-depth critique of Google Antigravity (Google's newly launched Agent-powered IDE platform) on Reddit, striking a chord with many users.

Google Antigravity Platform Background: Google Antigravity is Google's next-generation Agent-powered integrated development environment (IDE), designed to deeply integrate AI capabilities into the development workflow. By 'Agent-powered,' we mean the AI is no longer just a code completion tool, but an intelligent agent with autonomous planning, tool invocation, and multi-step execution capabilities. It can understand development intent, automatically break down tasks, invoke terminal commands, read and write files, and even perform code reviews. Antigravity integrates multiple mainstream large language models including Gemini (Google's in-house model) and Claude (developed by Anthropic), allowing developers to choose different models based on task characteristics. While this multi-model aggregation strategy theoretically combines the strengths of each provider, it also introduces new challenges around quota management and model switching costs.

As an AI programming environment integrating mainstream models like Gemini and Claude, Antigravity should be a productivity powerhouse, but this user's actual experience was filled with frustration.

What appears to be a simple complaint post actually reflects two core pain points of current AI programming tools: the mismatch between model capabilities and benchmark scores, and pricing/quota structures that are unfriendly to ordinary users.

High Benchmark Scores, But Frequent Real-World Failures

Performance Gap in Gemini Series

The user explicitly stated that Gemini 3.1 Pro's performance was "disappointing," while Gemini 3.7 Flash "seems to be guessing answers, lacking a reasonable chain of thought" and often "half-asses everything."

What puzzled him most: These models achieved very high scores on benchmarks yet frequently failed at the simplest tasks. This is actually a long-standing industry controversy — the gap between benchmark scores and real development scenarios.

The Mismatch Between Benchmarks and Actual Capability: AI model benchmarks typically use standardized datasets like HumanEval (code generation), MMLU (multitask language understanding), and GSM8K (mathematical reasoning). These test sets have clear questions and standard answers, making quantitative comparison convenient. However, real development scenarios are far more complex than benchmarks: requirements are often vague and changeable, codebases have technical debt and implicit dependencies, and context may span dozens of files. A model's high scores on closed test sets may stem from overfitting to specific question types, or from training data containing samples similar to the test set (data leakage issues). This 'test-taking ability' cannot guarantee generalization performance in open environments — like students who score high on practice tests but struggle in real projects. Multiple industry studies have shown that the correlation between benchmark scores and user satisfaction is far lower than expected.

Many models optimize for specific question types to achieve impressive numbers on standardized test sets, but this optimization doesn't always transfer to actual programming tasks with complex contexts. When developers face vague requirements, implicit constraints, and multi-file dependencies in real projects, the model's "test-taking ability" falls short.

Extreme Dependence on Prompts

The user also mentioned that these models "require extremely detailed prompts, unlike other models," and even when he provided well-formatted prompts, the output was still "hard to trust" and "most of the time it screws things up."

This exposes the insufficient instruction-following robustness of some current models.

Technical Analysis of Instruction-Following Robustness: Instruction-following robustness measures a model's performance stability when facing incomplete, ambiguous, or non-standard prompts. Excellent AI assistants should have 'intent inference' capability — completing reasonable assumptions from brief descriptions or clarifying requirements through follow-up questions. This depends on the quality of the model's in-context learning and instruction tuning. If a model overly relies on detailed few-shot examples or verbose prompt templates, it indicates insufficient generalization ability. This may stem from training data biased toward high-quality, structured inputs, causing the model to struggle with colloquial, abbreviated real-world instructions. Ideal robustness requires reinforcement training on diverse, noisy instruction data and introducing feedback mechanisms for the model to learn proactive clarification.

Ideal AI programming assistants should have some intent inference capability, able to fill in reasonable context from brief descriptions. If users need to write lengthy prompts every time just to get the tool to barely work, the time saved by the tool is offset by the cost of writing prompts.

Quota Anxiety: One Command Burns Half the Allowance

Claude Model's High Token Consumption

If model capability is an "experience problem," then quotas are a direct "survival problem." This user described an extreme case: He simply had Antigravity run a PowerShell command, and it consumed 50% of his 5-hour quota.

This number is quite shocking. Executing a simple system command shouldn't consume such an enormous amount of tokens.

Token Consumption Mechanism in Agent Mode: AI workflow in Agent mode differs drastically from traditional single-turn Q&A. Executing a simple command may trigger complex internal loops: First, the Agent reads current context and task description (consuming input tokens); next, it generates an execution plan and invokes tools (like file read/write, terminal execution), with each tool call result becoming new context fed back into the model (consuming tokens again); the model verifies results, determines whether error correction or next steps are needed; finally generates a summary report (output tokens). If the Agent falls into inefficient 'think-execute-verify' loops, or repeatedly retries due to misunderstanding, token consumption grows exponentially. While Claude models have large context windows (up to 200K tokens), single-call costs are also higher. A single command consuming 50% of a 5-hour quota likely means the operation triggered dozens of internal interactions, with total tokens reaching tens or even hundreds of thousands.

This likely involves Agent mode's working mechanism — AI repeatedly reads context, plans steps, invokes tools, and verifies results during task execution, with each interaction round consuming tokens. When Agents fall into inefficient loops or over-"think," quotas get burned quickly.

Claude Model's Cost Structure: The Claude series (developed by Anthropic) is renowned for its excellent instruction-following and safety, but calling costs are also industry-high. Taking Claude 3.5 Sonnet as an example, API pricing is approximately $3 per million input tokens and $15 per million output tokens (actual prices vary by version and channel). In Agent mode, input tokens dominate due to repeated reading of large amounts of context. A seemingly simple task may involve reading core files from an entire codebase (easily tens of thousands of tokens), multiple rounds of tool calls and result verification, ultimately accumulating hundreds of thousands of tokens. For free users or student plans, quotas are typically billed by 'hours' or 'number of requests' rather than pure tokens, but still map to token consumption underneath. Once a single task triggers high-frequency interactions, quotas quickly bottom out. This is why many AI programming tools strictly limit student users — cost pressures are real.

Economic Pressure on Students

The user specifically emphasized his identity: "As a student, I don't have money to buy Claude subscriptions or any other paid plans." He also mentioned installing a tool called Ponytail hoping to reduce token consumption, but wasn't sure if it actually worked.

This touches on the core contradiction in AI tool adoption. High-quality models (especially the Claude series) have high calling costs, while free quotas are often depleted after a few experiments. For students, independent developers, and beginners, this creates a real barrier — they are precisely the group most in need of AI-assisted learning, yet least able to afford ongoing subscription fees.

Industry Signals from This Complaint

AI Tool Evaluation Needs to Return to Real Scenarios

This user's experience reminds us that when evaluating an AI programming tool, we can't just look at official benchmark scores and demo videos. The real development workflow — including model stability, tolerance for vague instructions, and actual token consumption efficiency — are key factors determining user experience.

Of course, a single Reddit user's complaint has subjective elements, and we should note their observations may be influenced by specific projects, network environments, or tool configurations. But when more and more users report similar issues, it's worth developers taking seriously.

Sustainability Challenges in Pricing Models

For platforms like Antigravity that aggregate multiple models, designing a pricing and quota mechanism that both covers costs and is friendly to ordinary users is a long-term challenge. Overly aggressive token consumption will directly discourage potential users, especially in a landscape with many competitors and low switching costs.

Practical Advice for Similar Users

For students and budget-limited developers facing similar struggles, here are some reference strategies:

• Define task boundaries: Before having the Agent execute tasks, break them down into small steps as much as possible to avoid it repeatedly looping and consuming tokens on complex tasks.
• Use free or low-cost models wisely: For simple tasks, prioritize models with lower consumption, saving Claude and other premium model quotas for truly complex scenarios.
• Monitor token consumption: Pay attention to quota changes after each operation, identify "token-hungry" operations and avoid them.
• Explore open-source alternatives: Locally deployed open-source models, while having capability limits, have no quota anxiety and are suitable for daily practice and learning.

Local Deployment of Open-Source Models: Open-source large language models (like Meta's Llama series, Mistral, DeepSeek-Coder, etc.) allow developers to deploy and run on local hardware, completely eliminating quota and network dependencies. The core challenge of local deployment is hardware requirements: 7B parameter models need at least 16GB VRAM (like RTX 4080), 13B models need 24GB+ (like RTX 4090 or A5000), and larger models require multi-GPU parallelism or quantization compression (4-bit quantization can halve VRAM requirements). The deployment toolchain is quite mature, with tools like Ollama, LM Studio, and vLLM providing one-click installation and API services. While open-source models still lag behind GPT-4 or Claude in complex reasoning tasks, their performance in code completion, documentation generation, and simple refactoring scenarios is acceptable. For students in learning stages, local models allow unlimited trial-and-error, accumulating prompt engineering experience without worrying about 'running out of quota and getting stuck.'

Conclusion

This real complaint from Reddit is a valuable footnote in the maturation process of AI programming tools. Technical promotions often focus on "what it can do," while ordinary users' pain points lie in "is it stable, is it expensive, is it easy to use." For Antigravity and its team, what truly wins developer trust is never the numbers on benchmark leaderboards, but delivering reliable results in one real task after another.

Key Takeaways