Hardcore Test of 20 AI Coding Tools: Cursor and Codex Take the Top, Why Do Trae and Others Fail?

The AI coding tool market is expanding at a visible pace, but which tools can truly shoulder the burden of production environments? A Bilibili creator used a seven-dimension scoring matrix to conduct a hardcore ranking of 20 mainstream AI coding tools, from "solid" to "lacking." This list not only provides conclusions but also exposes the core differentiation logic of current AI coding tools—some have already approached human engineers in solving complex engineering problems, while others are still circling within specific scenarios.

Scoring Framework: Seven Dimensions Determine a Tool's Fate

The scoring system of this list is not arbitrary but based on seven weighted dimensions: Capability (30%), Consistent Reliability (20%), Ruggedness (15%), Durability (10%), Cost Controllability (10%), Domestic Availability (10%), Model Experience (5%). Each dimension has a clear definition and scoring basis, attempting to restore the comprehensive performance of tools in real development scenarios.

Capability: A Direct Test of Hard Coding Strength

Capability carries the highest weight at 30%, focusing on coding accuracy, multi-file refactoring ability, and architectural understanding. Scoring is based on public benchmark results and community consensus—without public results, it is hard to score high. This means whether a tool can correctly understand the original project structure and not lose its footing when modifying multiple files simultaneously is the first threshold distinguishing good from bad.

Consistent Reliability: Will Long Tasks Go Off Track?

Consistent Reliability (20%) focuses on the AI's performance in tasks spanning dozens of consecutive steps. For example, migrating an old codebase from Framework A to Framework B and passing all tests—will the tool "go off track" midway (industry slang meaning drifting further from the goal), and can it self-detect and correct errors? Scoring mainly looks at community reputation and whether the tool's source code includes a plan-checking mechanism (such as checkpoints).

In the context of AI programming, "going off track" usually refers to an agent gradually deviating from its original goal when executing multi-step tasks, such as introducing irrelevant modifications during refactoring, getting stuck in loop debugging, or forgetting early constraints. To mitigate this, modern tools often introduce a "checkpoint" mechanism: saving state after key steps and reviewing the plan, enabling the model to self-verify or roll back. This design draws on the credit assignment idea in reinforcement learning, guiding long-sequence decisions through intermediate reward signals to improve task completion rates.

Ruggedness: Survival in Extreme Engineering Environments

Ruggedness (15%) simulates five harsh scenarios: large repositories over 100,000 lines, weak network or disconnection environments, legacy messy code, Windows path compatibility, and API rate limiting. Scoring takes the best two performances among the five scenarios, as it is hard to demand a tool to cover everything. This dimension directly targets pain points in engineering practice—in domestic development environments, Windows compatibility and network stability are often key to whether a tool can be deployed.

Durability and Cost: Sustainability Behind the Tool

Durability (10%) assesses whether the company or open-source team behind the tool will go bankrupt, whether the business model is sustainable, and version stability. Large companies like Microsoft and Google naturally have an advantage here. Cost Controllability (10%) examines whether there is a free quota, whether BYOK (Bring Your Own Key) is supported, and whether local models can be loaded—this directly relates to the marginal cost of long-term use.

BYOK (Bring Your Own Key) is a mode that allows users to access tools using their own LLM API keys. Its core value lies in decoupling tools from model providers: users can freely choose more cost-effective models, avoid being locked into a single service, and facilitate enterprise cost accounting and compliance management. For developers who need long-term high-frequency use, BYOK combined with local model loading capability can significantly reduce marginal costs and network dependence, making it an important indicator for evaluating tool sustainability.

Domestic Availability and Model Experience: The Hard Threshold for Localization

Domestic Availability (10%) is judged from four dimensions: whether a proxy is needed, whether payment is convenient, Chinese-language scenario performance, and enterprise compliance. Proxies not only add costs but also carry account suspension risks, and uploading code to overseas servers is a red line for enterprise compliance. Model Experience (5%) is highly subjective, examining onboarding pleasantness, user stickiness, and even "quirks," and is the only dimension allowing emotional scoring.

Enterprise compliance in AI coding tool selection often involves data export and code security. Uploading source code to overseas servers may violate the Data Security Law or industry regulatory requirements, especially constituting a red line for sensitive sectors like finance and government affairs. Therefore, domestic availability is not just about network latency but also legal risks. Tools that support private deployment or domestic nodes can effectively avoid such issues, which is why this carries higher weight in localization assessment.

T0 Tier: Claude Code and Codex Tie at the Solid Level

The highest-scoring T0 level is named "Solid," with only Claude Code and Codex selected, both scoring 8.4 points without a clear winner. Their common trait: Capability and Consistent Reliability are near perfect (some sub-items close to 10), meaning they have reached a level ready for direct production use in coding accuracy and long-task reliability. Both also perform well in Ruggedness and Durability, although Cost Controllability and Domestic Availability are not their strengths.

This result largely aligns with community consensus—Claude Code and Codex have consistently led in reputation for complex refactoring and long-chain tasks, but high usage costs and network thresholds deter many domestic developers.

Using AI coding tools directly in production typically means the tool needs high code accuracy, predictable behavior, and error recovery. In the industry, this is often measured by "human engineer reviewability": AI-generated code should pass existing test suites, and modification scope should be controllable. Long-task reliability requires tools to maintain contextual consistency across dozens of interactions, avoiding cumulative errors. Tools at this level can significantly reduce manual review burden, but code review is still recommended at this stage.

T1 Quasi-Top: Cursor and Gemini Close Behind

Cursor (7.8 points) and Gemini (7.5 points) form the "quasi-top" tier, meaning "usable in production but with a gap from the Solid level." Cursor has the highest user stickiness among all tools with outstanding Model Experience, but slightly lags T0 in Consistent Reliability and Ruggedness. Gemini enters with Google's Durability advantage and balanced performance. The gap between the two is mainly reflected in Capability sub-items—multi-file refactoring and architectural understanding remain less stable than Claude Code and Codex.

T2 Main Force and T3 Average: Daily Usable but Lacking Core Competitiveness

The T2 tier is called "Main Force," mostly scoring around 7, including Kimi, Kling, Tongyi Lingma (the original "Qianwen" should be Tongyi Qianwen), OpenCloudTrain, Copilot (original "CoupleLauter"), Adder, DeepSeek^Ad Harness, etc. These tools are basically sufficient for daily use but always lack one core competitiveness in Capability or Consistent Reliability, preventing them from reaching the top.

The T3 tier is defined as "so-so, always has some issues when used," including Continue, Zed, Baidu's Comet, WorkBody, and GLM (synonymous with Lingma). Among them, WorkBody has been popular recently, but since its positioning leans toward AI office documents and collaboration rather than professional programming, it scores low in coding categories.

T4 Lacking Tier: Only Two Tools Deemed Not Recommended

The bottom T4 "Lacking" tier includes only two tools (not specifically named in the original). The creator said if the list expanded to 30 or 40 tools, the Lacking tier could list 10 or more. Tools in this tier are basically not recommended for any production scenario.

Insights Behind the Scores: No All-Round Player

From the weighted matrix, no tool scores high in all dimensions. Claude Code and Codex win on Capability and Reliability but are dragged down by cost and domestic availability; Cursor wins on Model Experience and user stickiness but falls slightly short in hard engineering strength; domestic tools like Tongyi Lingma and DeepSeek Harness have clear advantages in domestic availability and cost but still need to catch up in Capability and Durability.

The value of this list lies not in the ranking itself but in providing a reusable evaluation framework. Developers can find the most suitable tool in the matrix based on their own scenarios—whether pursuing extreme refactoring capability or prioritizing compliance and cost. The creator will subsequently provide detailed breakdowns of each tool, which is worth following.

One-line Summary

Seven-dimension scoring matrix ranks 20 AI coding tools; Claude Code and Codex tie at T0.

Paragraph Summary

The article introduces a Bilibili creator's ranking of 20 mainstream AI coding tools using a seven-dimension weighted scoring matrix. Dimensions include Capability (30%), Consistent Reliability (20%), Ruggedness (15%), Durability (10%), Cost Controllability (10%), Domestic Availability (10%), and Model Experience (5%). Results show Claude Code and Codex tie at T0 "Solid" with 8.4 points, near-perfect in Capability and long-task reliability; Cursor and Gemini rank T1 quasi-top; T2 includes Kimi, Tongyi Lingma, and other daily-usable tools; T3 is average; T4 has only two not recommended. The article notes no all-round player and emphasizes the framework helps developers choose tools by scenario.