Running More Agents Doesn't Mean More Output: Verification Is What Really Determines Efficiency

Open five Agent windows at once and watch them each scroll their output, and it's easy to get the illusion that "I can get five things done today." This Bilibili video makes a counterintuitive point: no matter how many Agents you open, all you're scaling is the volume of changes—what actually makes results better is verification.

More changes never equals more features

The video puts it bluntly: the internet circulates the claim that "big tech companies have tens of thousands of Agents working every day," but tens of thousands of Agents doesn't equal tens of thousands of new features.

This was the rule back when humans wrote code, too. Behind a single new feature trails a long string of patches, tests, and edits to old code—hundreds or thousands of changes in a day, of which maybe only one actually makes it into others' hands. Agents accelerate this whole process; changes become cheaper and denser, but the conversion rate from "change → usable feature" doesn't automatically rise along with it.

Increasing the number of Agents is essentially increasing the number of candidate solutions. And candidates themselves carry no value judgment—they're just "possibly correct" things.

Five windows each produce a version—how would you pick?

A natural idea: have five windows each produce a version, and pick the best one.

The problem lies in the act of "picking." The video points out that in practice, what you're mostly evaluating is "which version reads more smoothly," while which version actually works depends entirely on how you check it. This shifts the problem from "generation capability" to "verification capability"—whether you have a reliable sieve determines whether you can sift out the truly valuable version from these five.

Without a sieve, five candidate versions are just five things of unknown correctness, and picking degenerates into going by feel.

The AlphaCode lesson: no matter how many candidates, you still need tests to sift round after round

The video cites the public case of DeepMind's AlphaCode: first generate a large number of candidate solutions, then sift them with tests round after round—the vast majority fail the tests, and only the few that remain are eligible for submission.

The key to this process isn't "generating a large number of candidates" but "sifting with tests round after round." The scale advantage of the generation stage can only be converted into deliverable output when paired with a sufficiently strict filtering mechanism. Conversely, if the sieve is too loose, the more you generate, the more bad things slip through.

This also explains why simply piling on more Agents yields limited results: you're scaling the supply side, while the acceptance criteria on the demand side haven't gotten any stronger in sync.

AlphaCode is a competitive programming model publicly released by DeepMind in 2022. Its core idea is to first sample and generate tens of thousands of candidate programs at scale for a given problem, then use the problem's own public test cases as a first round of filtering, further cluster and filter the passers using additional hand-constructed inputs, and finally submit only a very small number for official evaluation. The paper reports that it reached roughly the top 54% of human competitors in Codeforces contests. This process is often cited as the representative of the "large-scale sampling + strict filtering" paradigm: the model's per-generation accuracy isn't high, but by expanding the candidate pool and pairing it with efficient filtering, the overall success rate is substantially raised. Notably, the payoff of this paradigm depends heavily on the reliability and cost of the filtering stage—the stricter and cheaper the filtering, the more the advantage of sampling scale can be realized.

The sieve itself leaks

The video makes a further observation: the sieve itself leaks. Write your tests loosely, and bad things can stride right through.

This sentence is worth mulling over repeatedly—all tests passing only means it passed the tests you wrote, not that it's actually correct. The quality ceiling of verification depends on the quality of the verification standard. Using a loose set of tests to screen Agent output only yields the illusion of "looks like it passed."

So the real direction for improvement isn't "open more windows" but "make every verification more accurate." The example the video gives is small but practical: for an AI-summarized weekly report, first check the totals against last week's—that's a small sieve, low-cost, yet it can stop obvious errors.

Test completeness is a classic hard problem in the software testing field. Dijkstra famously said: testing can only prove the existence of defects, not their absence. That is, any set of test cases is only a finite sampling of the input space; uncovered branches, boundary conditions, and exception paths remain blind spots. This problem is especially prominent for Agent output, because the generated code or text often "looks reasonable"—syntactically correct and stylistically consistent—yet errs on semantics or business rules. Common ways to improve sieve quality include: constructing acceptance cases item by item based on requirements, introducing mutation testing to assess the detection capability of the test set itself, and cross-checking against standards independent of the generator.

Make the sieve small, accurate, and built into the process

Following the logic of this video, in practice this comes down to a few points:

• Give each type of Agent output a clear acceptance checklist. Not "looks fine," but specific to verifiable numbers, fields, and boundary conditions.
• Use small sieves for upfront interception. Low-cost checks like verifying weekly report totals against the sum are well suited to the very front of the process, blocking low-level errors first.
• Beware the false sense of security from "all passed." Test pass rate is a function of sieve quality, not output quality. If the sieve is loose, no matter how high the pass rate, it proves nothing.
• Before scaling quantity, first ask whether the sieve can keep up. If your existing verification methods can't support five concurrent streams, opening more windows will only multiply the review burden.

The question posed at the end of the video is well suited to daily self-checking: changes are getting cheaper and cheaper, and what's valuable is that sieve. The few things you accepted today—which sieve did they pass through?

There's no standard answer to this question, but it pulls attention back from "how many Agents did I use" to "what standard do I accept by." In an environment where Agent output is getting cheaper and cheaper, the latter is the scarce capability.

One-line Summary

What the number of Agents scales is the volume of changes—what really determines output quality is the verification standard.

Paragraph Summary

The article points out that opening multiple Agent windows at once only multiplies the volume of changes and doesn't automatically produce usable features. When multiple candidates are laid out before you, the quality of your selection depends on whether you have a reliable verification standard, not on generation capability itself. The DeepMind AlphaCode case shows that large-scale candidates must go through strict, round-after-round test filtering before the few qualified ones have delivery value. Furthermore, the sieve itself leaks: all tests passing only means the output satisfies the tests you wrote, and loose verification only brings the illusion of "looks like it passed." So the direction for improvement isn't opening more windows, but making the acceptance checklist small, accurate, and built into the process—confirming your verification methods can hold up before scaling quantity.