Claude Code Ultra Review Hands-On: Deep Dive into Multi-Agent Code Review

What is Ultra Review?

Claude Code is about to launch a brand-new feature called Ultra Review — a major upgrade to the existing /review command. Unlike standard code review, Ultra Review spins up multiple independent sub-Agents in the cloud, spending 10-20 minutes conducting an in-depth code audit that not only finds bugs but also verifies whether those bugs actually exist.

The feature is currently hidden behind a Feature Flag and hasn't been rolled out to all users yet. Feature Flags are a standard release practice in modern software engineering, allowing teams to dynamically control feature visibility without modifying code. Through Feature Flags, Anthropic can perform granular canary releases of Ultra Review: first to internal employees, then to specific paid tiers, and finally to all users. This mechanism also naturally supports A/B testing — showing different Agent configurations to different user groups and using data to drive decisions on optimal parameter combinations.

Through reverse engineering of the Claude Code binary, the feature's internal codename is revealed to be "Bug Hunter," defaulting to a "fleet" of 5 sub-Agents with a maximum expansion to 20 (likely for enterprise users).

Ultra Review's Four-Phase Workflow

Setup Phase: Initializing the Review Session

When a user executes the /ultra review command and specifies a PR, the system launches a review session on Claude Code's cloud version. Under the $200/month plan, 3 Ultra Reviews are currently available for free.

Find Phase: Multi-Agent Parallel Search

This is one of Ultra Review's most core innovations. The system launches 5 independent sub-Agents, each starting from different locations in the codebase and analyzing recent code changes along different paths.

The multi-Agent parallel architecture stems from distributed AI system design principles. In large language model applications, a single Agent is limited by context window size (typically 100K-200K tokens) and often struggles with very large codebases. Multi-Agent architecture breaks through single-context limitations by decomposing and parallelizing tasks, while also leveraging the concept of "ensemble learning" — similar to Ensemble methods in machine learning, where the combined result of multiple independent judgments often outperforms any single judgment.

There's a deep technical rationale behind this design: the order in which code is loaded into the context window affects bug visibility — a bug that's obvious in one ordering might be overlooked by the model in another. This phenomenon is known in academia as "Positional Bias." Research shows that LLMs tend to pay more attention to the beginning and end of input sequences, with information in the middle being easily "forgotten." In code review scenarios, the same bug placed at different positions in the context can see detection probability vary by several times. Multiple Agents approaching from different angles is a systematic way to combat this positional bias, significantly improving bug detection rates.

Additionally, these sub-Agents likely have different "personas" — one focused on billing logic, another on security, and so on. In the tested PR (a voice calling feature with approximately 11,000 lines of new code), the Find phase discovered 64 potential bug candidates.

Verify Phase: Independent Verification to Filter False Positives

This is the biggest differentiator between Ultra Review and traditional code review tools. False positives are a long-standing core pain point for static code analysis tools. Well-known industry tools like SonarQube and Coverity typically have false positive rates between 30%-70%, causing developers to develop "Alert Fatigue" and gradually ignore tool output. Internal Google research shows that when a tool's false positive rate exceeds 10%, developer adoption rates drop sharply.

The Verify phase introduces independent verification Agents that check each bug candidate from the Find phase one by one, confirming whether they actually exist. In essence, it simulates a manual secondary confirmation process, combining AI's high Recall with the verification step's high Precision, attempting to solve this industry-wide problem at the automation level. In testing, the verification phase rejected at least 9 false positive bugs. This mechanism effectively prevents Claude Code from making unnecessary modifications to non-existent issues.

Dedupe Phase: Deduplication and Merging

Since multiple sub-Agents may discover the same bug from different angles (just with different descriptions), the Dedupe phase merges duplicate findings into unique issues, ensuring the final report is concise and clear.

Ultra Review vs Standard Review Comparison

Running both the standard /review and Ultra Review on the same 11,000-line PR reveals significant differences:

Dimension	/review	Ultra Review
Runtime	3-4 minutes	~17 minutes
Execution location	Local	Cloud
Verification step	None	Yes
Review style	Quick audit, flags deviations	Attacker perspective, deep tracing

Through GPT Codex's comparative analysis of both reports:

• Standard Review is more like a quick audit of the entire codebase, flagging anything that deviates from convention
• Ultra Review is more like an attacker — it picks a path through the PR and tries various ways to break it

Ultra Review discovered race conditions and lifecycle bugs that the standard Review completely missed, and performed better at cross-file correlation analysis. Race conditions are among the hardest bugs to find in concurrent programming, occurring when multiple execution threads or processes access shared resources and program behavior becomes abnormal due to non-deterministic execution order. These bugs almost never reproduce in single-threaded testing and only appear sporadically in high-concurrency production environments. Traditional static analysis tools struggle to detect them because they cannot simulate runtime state. Ultra Review's ability to find race conditions demonstrates that its multi-Agent deep tracing mechanism possesses cross-file, cross-call-chain semantic understanding rather than simple pattern matching.

Applying the Verification Pattern to Your Own Workflow

Even without access to Ultra Review, its core design pattern — "Find + Verify" — can be fully replicated in your own workflow. Based on this approach, you can create a custom Fleet Review Skill:

1. Find phase: Simultaneously launch 3 Claude Code sub-Agents and 3 Codex CLI (Headless mode) sub-Agents to search for bugs
2. Verify phase: Use Claude Code verifiers and Codex verifiers for cross-validation

This cross-validation is particularly valuable — sometimes Codex determines that a bug found by Claude Code isn't actually a bug, and vice versa. Verification by two different models significantly improves result reliability. This approach is directly aligned with the principle of Model Ensemble in machine learning: models with different architectures or training data tend to have complementary error patterns, and combining them can systematically reduce any single model's blind spots.

Usage Recommendations and Outlook

Based on current information, the recommended code review strategy can be tiered by importance:

1. Daily PRs: Use /review for quick review (3-4 minutes)
2. Important features: Additionally use Codex for supplementary review
3. Critical/large PRs: Use /ultra review for deep review (10-20 minutes)

Ultra Review currently offers only 3 free uses per month, suggesting high operational costs. Anthropic may be using different model configurations in the cloud, possibly even mixing unreleased models. In the future, the feature may undergo A/B testing with different sub-Agent configurations and prompt combinations.

From a broader perspective, Ultra Review represents an important trend in AI code review: moving from single-Agent one-pass scanning toward multi-Agent collaboration with multi-phase verification for deep review. This "fleet-style" review architecture, combined with independent verification mechanisms, is redefining the quality ceiling of automated code review.

Key Takeaways

• Ultra Review uses a four-phase workflow (Setup→Find→Verify→Dedupe), defaulting to 5 sub-Agents searching for bugs in parallel from different angles
• The independent verification phase is its core innovation, effectively eliminating false positives and preventing unnecessary code changes for non-existent bugs
• Compared to standard /review, Ultra Review can discover deep bugs like race conditions, but takes longer (~17 minutes vs 3-4 minutes)
• The feature is internally codenamed Bug Hunter; the $200/month plan currently offers only 3 free uses, suggesting high operational costs
• Its "Find + Verify" design pattern can be replicated through custom Skills, with cross-validation between Claude Code and Codex yielding even better results