[KongchangAI]
· 2 min read· 1,393 words

Four Models, Each With a Role: An AI Division-of-Labor Practice Revealed by Logs

Four Models, Each With a Role: An AI Division-of-Labor Practice Revealed by Logs

Multi-model division-of-labor experiment: Grok cheap review, Codex deep code reading, Tīrtha routine setup, ledger quantifies real effects.

The article introduces a multi-model division-of-labor workflow: Claude plans and checks, Grok launches low-cost adversarial review, Codex deeply reads the codebase, and the self-developed router Tīrtha takes on routine setup. The author uses a continuously maintained ledger to quantify the effects: of Grok's 519 findings, 309 held up, 207 were absent from Claude's first draft, median cost per run about 2 cents; Codex had only 50 findings but 33 were new, once finding that the online gateway was not writing usage records, causing billing blindness, at the cost of a median consumption of about 84,000 tokens per run and multiple limit triggers. Tīrtha has just completed a smoke test. The ledger has methodological flaws such as backfilling and model self-evaluation blind spots, and whether the overall team is better than a single model has not yet been verified.

Most developers are used to picking one model to handle everything. But when a single model has to both do the work and check its own output, a problem arises—it is precisely the worst at examining its own blind spots. A few months ago, the author's team began experimenting with letting different models each play their own role, and used a continuously maintained ledger to quantify the real effects of this division-of-labor experiment.

Log analysis of four models each playing their own role

Division-of-Labor Architecture: Who Does What

The allocation logic of this workflow is clear: Claude is responsible for planning and checking, Grok is responsible for launching "attacks" on what Claude has just written, Codex builds large modules, and the team's self-developed router Tīrtha takes on routine setup tasks.

The original design intent was not complicated—subscription quotas always hit their limits, and having the same model both produce and self-check is equivalent to putting the weakest link in the quality-control position. Splitting responsibilities both relieves quota pressure and lets each model leverage its own strengths.

Tīrtha is the author's team's self-developed model router; its name comes from Sanskrit, meaning "ferry crossing" or "passage." In this workflow, it does not directly participate in code generation or review, but instead distributes requests to different backend models according to task type. The router itself usually contains no model weights, relying instead on rules or lightweight classifiers to decide "who does it." This design allows the team to flexibly replace or add/remove underlying models without changing the upper-layer invocation method. Tīrtha taking on routine setup tasks is precisely about using its routing capability to direct simple, repetitive work to low-cost models, thereby preserving quota for high-cost models.

Grok: The Cheap Second Opinion

From September 25 to October 17, Grok raised a total of 519 findings. After manual review, 309 held up, of which 207 were content entirely absent from Claude's first draft, and 59 were critical and entirely new issues.

Of course it also makes mistakes: 28 findings were pure false positives, and another 30 errors stemmed from missing information in the task brief itself. But at a median cost of about $0.021 per review (the most recent 30 cost $2.32 in total), a reviewer that occasionally errs is still excellent value.

The reason Grok is cheap is that it never touches the full codebase. It only receives a short brief of about 2,300 tokens—containing claims and evidence—and its task is to "tear it down." The author gives an example from that day: a gateway fix built by Claude passed 55 tests, while a 17-cent Grok attack found three cases that would cause correct answers to be cleared. All three were real issues, and all have now been fixed.

Median cost refers to the value in the middle after arranging all individual review costs by size, rather than the average. Compared with the average, the median is insensitive to extreme values and more robustly reflects "how much a typical review costs." The article mentions that Grok's median cost per review is about $0.021, meaning half of the reviews cost less than this and half cost more. This measure is especially useful in scenarios with large cost fluctuations, avoiding a few expensive runs inflating the overall impression.

Codex: The One That Reads Deepest

Codex raised far fewer findings, only 50, but 39 held up, and 33 were new to Claude. The most valuable one was: it found that the online gateway under certain requests was not writing usage records at all, leaving billing completely blind. This problem is absolutely invisible without reading the code.

Depth comes at a price. Codex's median consumption per run is about 84,000 tokens, and between September 22 and October 14 it triggered the usage limit 7 times, two of which made it unusable for the entire day. Every time a routine task is handed to Codex, it consumes window quota reserved for hard problems.

Token consumption is one of the core metrics for measuring the cost of using large models. When a model processes text, both input and output are split into tokens (roughly equivalent to one English word or half a Chinese character), and the total number of tokens consumed per call directly determines the cost and the occupation of rate limits. Codex's median consumption per run is about 84,000 tokens, far higher than Grok's roughly 2,300-token brief, which explains why it more easily triggers usage limits. Subscription services usually limit the total number of tokens by time window (such as every 5 hours or daily), and deep code reading quickly exhausts window quota, making it impossible to handle hard problems afterward.

Tīrtha: A New Attempt at Routine Setup

The author candidly admits limited knowledge in this part. This week Tīrtha took on its first real task through OpenCode, after previously fixing a bug where a tool call returned as plain text (0/2 sessions usable before the fix, 3/3 usable after). At present this is only a smoke test; a real report card requires weeks of real tasks to accumulate.

The bet behind it is straightforward: let Codex do only what only it can do, and hand routine setup to a cheap and validated builder, thereby extending the subscription's lifespan. The team ran a similar version in the summer—a free cheap tier completed 36 of 52 fleet tasks, and there were no Codex quota errors from June to August. The timeline matches, but too many other variables changed during the same period, so it cannot be called proof.

What Cannot Yet Be Proven

The limitations listed by the author are equally worth noting:

  • Whether the entire team is better than Claude working alone: the test has been set up but not yet run.
  • Whether moving setup work to Tīrtha really extends subscription lifespan will be measured starting now.
  • The ledger itself has flaws: most of the 746 records were backfilled afterward from saved reviews, and it was Claude that judged which findings already existed in its own first draft. Letting a model judge its own blind spots is precisely the kind of problem Grok would seize on.

This candor is itself part of the method—the value of division of labor is not in proving that some model is stronger, but in clearly seeing, in a quantifiable way, the real contribution and cost of each link. The question the author raises at the end points in the same direction: do you have one model do everything, or do you give each one a job? Has anyone measured whether the real problems caught by a cheap reviewer truly exceed the cost brought by its false positives?

Smoke test is a term in software engineering that refers to quickly verifying with the minimal operations whether the system's core functions are basically usable before formal comprehensive testing. It does not aim to cover all boundary conditions, only to answer "can it run." In the article, Tīrtha taking on its first real task and the number of usable sessions changing from 0/2 to 3/3 before and after the fix fall within the scope of smoke testing. Passing a smoke test only means the system has not completely failed; proving long-term stability and efficiency still requires weeks of real-task data accumulation.

One-line Summary

A multi-model division-of-labor experiment: Grok provides cheap review, Codex reads code deeply, Tīrtha takes on routine setup, and a ledger quantifies the real effects.

Paragraph Summary

The article introduces a multi-model division-of-labor workflow: Claude is responsible for planning and checking, Grok launches adversarial reviews at low cost, Codex reads the codebase deeply, and the self-developed router Tīrtha takes on routine setup. The author uses a continuously maintained ledger to quantify the effects: of Grok's 519 findings, 309 held up, 207 were absent from Claude's first draft, and the median cost per run is about 2 cents; Codex had only 50 findings but 33 were new, and it once found that the online gateway was not writing usage records, causing billing blindness, at the cost of a median consumption of about 84,000 tokens per run and multiple limit triggers. Tīrtha has just completed a smoke test. The ledger has methodological flaws such as backfilling and model self-evaluation blind spots, and whether the overall team is better than a single model has not yet been verified.

#multi-model division of labor#AI workflow#Grok review#Codex#Tīrtha router#token cost#smoke test
Share:

Related articles