The Big 20-Way AI Coding Tool Showdown: Copilot/Cursor/Claude — Who's Swimming Naked?

AI coding tools have become so numerous it's dizzying—but which ones are real productivity, and which are "productivity illusions"? A Bilibili creator used a scoring matrix to rank 20 mainstream AI coding tools from "solid" to "trash," with detailed scoring dimensions and weights. This article breaks down the logic behind that ranking based on the video content, combined with consensus from multiple communities.

Ranking at a Glance: Only Two in T0, Only Two in T4

The creator divided the 20 tools into five tiers: T0 "solid" (top-tier), T1 "near-top" (first-class), T2 "mainstay" (daily usable), T3 "average" (specific scenarios), T4 "trash" (not recommended).

• T0 (solid): Claude Code, Codex, both with an overall score of 8.4—no clear winner.
• T1 (near-top): Cursor (7.8), Gemini (7.5).
• T2 (mainstay): Kimi, Kilo, Tongyi Lingma (referred to as "Qianwen" in the original video), Open Cloud Train, Copilot (referred to as "Couple Lauter" in the original video), Adder, DeepSeek^Ad Harness, etc., mostly scoring around 7.
• T3 (average): Continue, Zed, Baidu Comet, Workbody, GLM (referred to as "Tongyi Lingma" in the original video), etc.
• T4 (trash): Only two listed, not specifically named.

You may not have noticed: although Workbody is very popular recently, its positioning leans toward AI office documents and collaboration rather than pure coding, so it scores low on coding dimensions. The creator also emphasized that if extended to 30 or even 40 tools, the T4 list would be longer.

Scoring Dimensions and Weights: Capability 30%, Model Only 5%

This ranking isn't arbitrary—it's based on weighted scores across seven dimensions:

Dimension	Weight	Definition
Capability	30%	Coding accuracy, multi-file refactoring, architecture understanding
Reliability (routine task reliability)	20%	Staying on track through dozens of consecutive steps, self-correcting errors
Toughness	15%	Performance in large repos, weak networks, bad code, Windows, rate limiting
Durability	10%	The company/community behind the tool's staying power and version stability
Cost control	10%	Free quota, BYOK, local model support
Domestic availability	10%	Proxy requirements, payment convenience, Chinese scenarios, enterprise compliance
Model	5%	Subjective experience: pleasantness to use, stickiness, quirks

Capability: Hard Power Based on Benchmarks + Community Consensus

Capability accounts for 30%, with the core being whether the code is correct and whether it can refactor across modules without breaking things. Scoring is based on public benchmark results while referencing consensus from AI and programmer communities. Tools without public results struggle to score high here.

Reliability: Staying on Track Through Long Tasks Is the Real Skill

"Reliability" refers to routine task reliability, weighted at 20%. A typical scenario: migrating old code from framework A to framework B and passing all tests—dozens of consecutive steps. Does the tool "wander off"? Can it self-detect and correct errors? Lacking benchmarks, the creator mainly looks at community reputation and whether the tool's source/architecture has plan/checkpoint mechanisms.

The plan/checkpoint mechanism is a key engineering approach for measuring long-task reliability. Plan means the tool generates an explicit plan before executing multi-step tasks, breaking big goals into verifiable sub-steps; checkpoint means saving state at key nodes so that if later steps fail or deviate, it can roll back to the most recent checkpoint and retry. Tools with such mechanisms are less likely to "wander off" in scenarios like migration and refactoring that span dozens of consecutive steps, and can self-detect and correct errors. The original article uses the presence of such mechanisms in source/architecture as one of the scoring criteria for the "reliability" dimension—that's why.

Toughness: Take the Two Highest Scores Across Five Scenarios

Toughness is weighted at 15%, examining five scenarios: locating modification points in large repos of 100k+ lines, weak network/disconnected/proxy-required environments, legacy bad code, Windows path and terminal compatibility, and error reporting/recovery under API rate limiting. Since no tool can do everything, scoring takes the two highest-scoring scenarios.

API rate limiting refers to model service providers setting caps on request counts or token usage per unit time, returning errors like 429 when exceeded. In AI coding tools, rate limiting directly interrupts long tasks: if the tool cannot recognize rate-limit signals, back off and retry, or fall back to a backup model, it manifests as freezing or task interruption. The ranking lists "error reporting and recovery under API rate limiting" as one of the five toughness scenarios, precisely to examine fault-tolerance design when quotas are constrained—especially practical for developers using free quotas or shared team keys.

Durability: Will the Company Fold? Is the Version Stable?

Durability is weighted at 10%, looking at the company or open-source team behind the tool. Big tech (Microsoft, Google) and large AI companies score higher; newly released preview versions (like DeepSeek Harness) lower scores due to unstable interfaces, but company strength can partially offset this.

Cost Control: Free, BYOK, Local Models

Cost control is weighted at 10%, assessing whether there's a free quota, whether BYOK (bring your own model key) is supported, and whether it can fall back to locally hosted private models. Local models can drive marginal costs toward zero.

Domestic Availability: Proxy, Payment, Chinese, Compliance

Domestic availability is weighted at 10%, with four checkpoints: whether a proxy is needed (proxies cost money), payment convenience, Chinese scenario performance, and enterprise compliance (whether code is uploaded to overseas servers). For domestic developers, this is often the deciding factor.

Model: Subjective but Resonant

The model dimension is weighted at only 5%—a highly subjective experience score: is it comfortable to use, is it "addictive," does it have quirks. This item easily resonates but has limited impact on overall ranking.

BYOK stands for Bring Your Own Key, meaning users bring their own LLM API key to access coding tools. In this mode, the tool itself doesn't resell model call quotas; instead, users pay directly to OpenAI, Anthropic, or local model providers. For developers, BYOK means cost transparency, model switching, and avoiding lock-in to a single tool; for enterprises, it facilitates unified procurement and auditing. The ranking groups BYOK with free quotas and local model support under the "cost control" dimension precisely because these three together determine the marginal cost of long-term use.

Top Performers by Dimension: Who's Strong Where

• Capability and reliability: Claude Code and Codex are near-perfect (some individual items near 10).
• User stickiness (model): Cursor scores highest.
• Durability: Copilot (Microsoft-backed) is highest.
• Domestic availability: Domestic tools like DeepSeek Harness and Tongyi Lingma stand out.
• T4 "trash": No standout performance in any dimension.

Public benchmarks here mainly refer to standardized datasets for evaluating code generation, such as HumanEval, MBPP, SWE-bench, etc. They provide comparable scores using unified problems and automatic judging, and are an important basis for the 30% weight of the "capability" dimension. But benchmark results have limitations: problems are mostly single-file or short-context, making it hard to cover real scenarios like 100k-line repo refactoring, weak networks, and rate limiting. Therefore, the original article also incorporates consensus from AI and programmer communities, and lowers scores for tools without public results. Understanding this is key to seeing why the ranking doesn't treat benchmark scores as the sole yardstick.

Conclusion: No Perfect Tool, Only Matched Scenarios

The value of this ranking isn't in determining who's #1, but in providing a reusable evaluation framework. Claude Code and Codex lead in capability and long-task reliability, suitable for production environments with high code quality demands; Cursor wins on experience and stickiness; domestic tools have an edge in domestic availability; and Workbody, positioned for office collaboration, scores low on coding dimensions—not because the tool itself is bad, but because of scenario mismatch.

The creator says they will later explain the scoring rationale for each tool in detail. For developers, rather than chasing rankings, it's better to choose the tool that best matches your core scenarios (large repo refactoring, weak network environments, compliance requirements, budget constraints).

(This article is compiled from a Bilibili creator's video content; scores and opinions belong to the original author.)