Companies Selling Tokens Are Now Scoring Your Coding Efficiency

A conversation circulating in developer communities has brought an increasingly common phenomenon to the surface.

In the conversation, an engineer is blocked by the "usage limit" of an AI coding assistant, and the system prompts him that his "efficiency needs improvement." A colleague asks what he did with the tool yesterday, and he says "mostly saying no"—the AI wanted to rewrite the entire payment service from scratch, and he only let it change one line. As a result, he ranks at the bottom of the efficiency and time-saved leaderboard. Meanwhile, the person at the top has the AI rebuild the entire application from scratch every morning, and the system shows he saves 40 hours per day.

The last sentence is the key point: the scoring rules for this leaderboard were written by the company selling tokens.

From "Assistant" to "Overseer": How the Scoring Mechanism Works

AI coding agents have spread rapidly in recent years, and it is not unusual for tool providers to introduce usage limits in subscription plans. But turning "efficiency" into a quantified metric—and then into a leaderboard—changes the nature of things.

According to the description in the conversation, the core metric of this scoring logic is "engineering hours saved per dollar spent." In other words, the more you let the AI generate code and the more tokens you consume, the more the system tends to consider you "efficient." Conversely, the more carefully you constrain the AI's output scope and let it make only minimal changes, the lower your score becomes.

This creates a directional incentive misalignment: the score rewards AI usage rather than engineering judgment. The engineer who changed only one line actually avoided a high-risk, large-scale rewrite—in scenarios like payment systems, minimal changes are often the correct engineering decision. But on the leaderboard, he ends up at the bottom in efficiency.

AI coding agent refers to an AI system that can autonomously plan and execute multi-step coding tasks, as distinct from traditional code completion tools (such as early Copilot's line-by-line suggestions). Typical examples include Claude Code, Cursor's Agent mode, and Devin. They usually work by receiving a natural-language goal, then reading the codebase, modifying multiple files, running tests, and iterating on failures. Because a single task may involve dozens of model calls and large context reads, token consumption is far higher than single completions—which is precisely why providers need to introduce usage limits. And because consumption is strongly correlated with "task scale," a scoring system that converts tokens into efficiency will naturally favor delegating larger-scale tasks.

Top and Bottom: Two Completely Different Kinds of "Efficiency"

The conversation describes both ends of the leaderboard. The person at the bottom: the AI asked to rewrite the payment service, and he approved only a one-line change. The person at the top: every morning he has the AI rebuild the entire application from scratch, and the system shows he saves 40 hours per day.

There are only 24 hours in a day. The figure "saves 40 hours per day" already exposes the problem with the metric—it does not measure time actually saved, but hypothetical work hours converted from the amount of code the AI generates. Rewriting the entire application sounds like massive output, but without corresponding verification, testing, and review, the actual value of that code is questionable.

In other words, the "efficiency" at the top of the leaderboard may well be synonymous with high consumption. And the "inefficiency" at the bottom comes precisely from cautious gatekeeping of AI output.

Token is the basic unit of text processed by large language models, roughly equivalent to a word or part of a word. APIs bill input and output tokens separately, so "how many tokens are consumed" directly corresponds to "how much money is spent." In coding agent scenarios, a task repeatedly sends codebase snippets, conversation history, and tool return results into the model, and input tokens often account for the bulk. The so-called "engineering hours saved per dollar spent" essentially converts the token bill into money, then divides by an assumed coefficient for "human work hours corresponding to the amount of AI-generated code." There is no unified standard for this coefficient; it is set by the platform itself, so both the numerator and denominator of the metric are subjective—which is why numbers like "saves 40 hours per day" can appear.

Who Defines Efficiency: The Core of the Conflict of Interest

The most insightful line in the conversation is this: the scoring rules are written by the company selling tokens.

This is not to say the tool providers necessarily have malicious intent, but a structural conflict of interest objectively exists. Token consumption is the revenue source for such companies, and a scoring system that rewards "using AI more" naturally aligns with revenue growth. The problem is that it is simultaneously packaged as an objective evaluation of developers' engineering ability.

When scoring standards are unilaterally set by an interested party, the leaderboard is no longer a neutral feedback tool but a form of behavioral guidance. It guides users to generate more and filter less, consume more and judge less—exactly the opposite direction from good engineering practice.

Conflict of interest in professional ethics means that when an evaluator's judgment may be affected by their own interests, the neutrality of the evaluation is unreliable. The classic handling principle is "recusal"—for example, an auditor may not audit a company in which they hold shares. AI tool vendors are in a similar position: they are both the maker of the scoring rules and the revenue recipient from token sales, as well as the provider of the product used by those being scored. The combination of all three makes it difficult for any "efficiency" metric to be regarded as third-party certification. The key to identifying this kind of conflict is not to assume subjective malice, but to acknowledge that even if the rule makers are entirely well-intentioned, as long as the direction of incentives is not fully aligned with user interests, the metric will drift over time in a direction favorable to the platform.

How Developers Should View Such Metrics

The engineer in the conversation responds to the AI's rewrite proposal by "saying no," which is itself an underrated ability. At a time when AI coding tools are becoming increasingly capable, knowing when not to use AI and how narrowly to constrain its output may be more important than having AI write more code.

When facing efficiency scores from platforms, several judgments are worth keeping:

• The score reflects how the platform wants you to use the product, not necessarily your real engineering efficiency.
• The estimated time saved needs to be examined in terms of its calculation basis, especially when the value exceeds physical possibility.
• In high-risk modules such as payments and security, the principle of minimal changes should take priority over any leaderboard.
• If the scoring system is directly tied to product revenue, it should be treated as a marketing signal rather than a capability certification.

Why this conversation resonates among developers is that it explains in very few words one thing: when a tool starts grading you, first see clearly how the people doing the grading make money.

The principle of minimal changes has a long history in software engineering and is in line with surgical change and the test-driven idea of "make it fail first." Its core assumption is that every change introduces risk: the larger the scope of change, the more paths need to be verified, and the larger the potential surface for regression defects. In modules such as payments, authentication, and accounting, a full rewrite means the entire trust boundary needs to be re-audited, whereas the impact of a one-line fix can be precisely delimited. This principle is in structural tension with the "inclination" of AI agents: generative models naturally tend to produce complete, coherent implementations rather than minimal patches. Therefore, when using AI, constraining the output scope often requires developers to actively impose it—and such constraints are precisely what a system scoring by output volume will not recognize.

One-line Summary

AI coding tool efficiency leaderboards score by token consumption, rewarding more generation rather than better judgment.

Paragraph Summary

Starting from a conversation circulating in developer communities, the article reveals the absurd phenomenon after AI coding agents introduced "efficiency scoring": an engineer who only lets the AI change one line ranks at the bottom for "saying no," while an engineer who has the AI rewrite the entire application every day tops the list for "saving 40 hours per day." The author points out that this metric, centered on "engineering hours saved per dollar," actually rewards token consumption rather than engineering judgment, and that time saved is merely a hypothetical value converted from code generation volume. More importantly, the scoring rules are written by the company selling tokens, creating a structural conflict of interest, and the leaderboard is more like behavioral guidance than objective evaluation. The article concludes with several judgment suggestions, emphasizing that in high-risk scenarios such as payments and security, the principle of minimal changes should be upheld.