Can AI-Written Specs Generate Code Directly? A Multi-Agent Review Workflow Explained

How risky is it to take a Spec generated by AI and use it directly to write code? The SDFlow workflow shared by Bilibili creator "Lao Dao Qian Li" offers one approach: after the spec is written but before coding begins, insert a multi-agent review gate that lets AI cross-examine the spec from multiple angles, while humans only make the key decisions. The core logic, practical details, and cost-benefit of this workflow are worth understanding for every engineer using AI-assisted development.

Why Does a Spec Need a "Review Pass"?

In a previous minimal development loop demo, the creator ran into a typical problem: the requirement was to alert when the temperature exceeds 80 degrees, but the spec's test tasks included a line saying "81.2 degrees should not trigger an alert." The sub-agent discovered the conflict while running tests, verified it, and confirmed the spec was wrong, then corrected it before continuing. This case exposes a key risk — if the spec is logically consistent from start to finish, tests will never catch the error, and the code will faithfully implement a wrong requirement.

Spec documents are large and complex, and manual review is burdensome. The creator cites an article on Martin Fowler's website noting that an author who tried three spec-driven development tools including SpecKit complained that the generated Markdown documents were "verbose and exhausting," and would rather review code instead. Tarek from the Claude Code team also admitted that he doesn't really read Markdown over 100 lines himself. Having AI do a review pass first becomes a pragmatic choice to reduce the manual review burden.

Multi-Agent Review: Fresh Context, Multiple Angles, Cross-Validation

SDFlow's review design references the G-Stick framework from YC president Gary Tan and extends it. The core approach has three points:

First, switch to fresh context. The session that wrote the spec no longer reviews it. Instead, independent agents are dispatched, each starting with a clean slate, seeing only the documents and code, unaffected by the thinking that went into writing it.

Second, review in parallel from multiple angles. Each angle gets an independent agent running simultaneously, including: CEO review (should this be done at all, and at what scope), engineering review (architecture error paths, test plans), developer experience review (are commands, interfaces, and configuration easy to get started with), code review (does the code in the spec actually work that way), domain review (going through a layered checklist by tech stack), and adversarial review (assume something will definitely go wrong, then find how). In addition, another model is asked for an independent second opinion — if you develop with Claude, ask the strongest model from Codex, and vice versa.

Third, dispatch on demand. Not every review runs every time. The fixed set includes CEO review, engineering review, code review, another model's review, and two adversarial reviews. The main session first assesses the requirement and design situation; 4 out of 16 scenarios are considered high-risk. If high-risk, additional reviews are dispatched: another model reviews again specifically for what the domain review missed, plus an extra adversarial review focused on overly optimistic estimates and boundary cases. Domain reviews are dispatched on demand based on whether the changes touch the backend, command interfaces, etc.

The value of adversarial review was demonstrated perfectly in a real case: the requirement was to notify on-call staff in a DingTalk group, with the push running in a separate goroutine to avoid slowing down temperature reporting. The adversarial review asked what happens if this push code crashes — in Go, a goroutine panic without recover causes the entire process to exit, so if the push crashes, the whole service is gone and temperatures can't be reported either. The spec didn't mention this, but two of the three adversarial reviews flagged it, the engineering review flagged it, and the Go domain review also caught it because this item was already in the checklist. The spec was ultimately amended to add: "Panics in the push must be recovered, errors logged, and the service continues running normally."

The G-Stick framework is an AI-assisted development methodology proposed by YC president Gary Tan (referred to as "Gary Teng" in the original). Its core thesis is "first have AI write a detailed spec, then have multiple AI agents review and iterate from different roles, and only then generate code." It emphasizes concentrating human judgment at key decision points while delegating tedious review, verification, and cross-validation to AI. SDFlow extends this with additional review roles (such as domain review and adversarial review) and a dynamic dispatch mechanism, making it better suited to real engineering scenarios.

Adversarial Review is a deliberate "fault-finding" review strategy: the reviewer first assumes the system will definitely fail, then works backward to deduce how the failure might occur and under which boundary conditions it would trigger. It doesn't aim to prove the solution correct but actively hunts for optimistic estimates, overlooked exception paths, and unstated assumptions. Compared with conventional review, adversarial review more easily exposes hidden risks such as goroutine panics, concurrency races, timeouts, and retries, so it is often used as an additional review for high-risk requirements.

From 50 to 18: AI Adjudicates First, Humans Only Make the Final Call

After several review passes finish, 50 issues are reported, some real and some not. The main session's process is: first merge duplicates and record which passes reported the same item — when multiple independent passes report the same item, confidence is higher. For example, the push panic item was reported by four passes. Then verify citations: every finding must include file line numbers and original text; check whether the file exists, whether the line number is correct, and whether that line really says what's claimed. Anything that fails verification is cut outright.

Passing citation verification only means the citation is real, not that the conclusion is correct. The main session then adjudicates item by item: will this problem actually occur during implementation? If yes, keep it; if no, cut it; if uncertain, leave it for a human. Cut items can't be silently discarded — a reason must be written. Where another model disagrees with the main session, both sides' arguments are handed to a human for judgment.

After this round, the 50 items were reduced to 28 after merging duplicates, and ultimately 18 were adopted to directly improve the spec document. Only one item required a human decision. The creator emphasizes that no humans are consulted mid-review; humans only see the final report. The report is presented as a webpage and always lists three questions: do you accept the scope, do you accept the dependencies and order, and do you accept the risks and countermeasures. Where there are two or more options, it must clearly state the options, which one is recommended, and what consequences each choice has for the system, for users, and for future development.

Citation Verification means, for each review finding, checking whether the referenced file actually exists, whether the line number is accurate, and whether the original text on that line matches the finding's description. This step filters out problems produced by AI hallucination — "plausible-sounding but unsupported by evidence." Passing citation verification only proves the citation is real, not that the conclusion is correct, so the main session still needs to adjudicate item by item whether "this problem will actually occur during implementation," leaving only uncertain ones for human judgment. This two-stage process of "verify facts first, then judge logic" significantly reduces the burden of manual re-checking.

Cost and Benefit: 17 Minutes and 15 Million Tokens Buy What?

This review run takes about 17 minutes and consumes over 15 million tokens, 90% of which are cache reads. The creator considers it worth it for two reasons:

Reducing LLM uncertainty. One team tested running the same sentence 1,000 times and got 80 different answers. The creator also ran his own experiment: opening three fresh sessions on the same spec, asking the same model only "review this spec yourself," about three minutes each time; across three runs, after removing duplicates, there were 16 items total, of which 7 were substantive, but only 2 were caught in all three runs, and 4 were caught only once. The push panic item, meanwhile, doesn't show up no matter how many times you run that same review method, yet four passes in SDFlow's multi-pass review reported it. An Anthropic article gave an example of reviewing code vulnerabilities: reviewing once each with several different prompts, what one pass misses another can catch. A paper in September this year had 10 models from OpenAI and Anthropic review requirement documents, running 100 times repeatedly; even the best-performing Anthropic expert found only about half the problems. The paper concluded that the misses were mainly due to inherent model shortcomings rather than randomness. Another paper in October also tested this: errors found by another model only partially overlapped with those found by the same model in a different session. So asking another model isn't because it's more accurate, but because it sees things differently.

The earlier a problem is found, the cheaper it is. This is an old conclusion in software engineering. A 2001 summary noted that fixing after delivery often costs 100 times as much as at the requirements/design stage, and even for smaller, less critical systems about 5 times. Writing code with AI makes rework faster, but you still have to re-test and re-review, and you only find out when something slips through to production. Fixing it before writing code is just a matter of changing a few lines. The creator does the math: a problem like the push panic, if only discovered after launch, means the entire service goes down; spending a dozen minutes finding it before writing code is worth it.

Review Blind Spots and a "Magic Spell"

Review is not omnipotent. In this run, the creator encountered something no one asked about from start to finish: should the time on-call staff see in the DingTalk group be UTC or Beijing time? Neither the interrogation nor the review asked, and UTC was hard-coded into the spec. This kind of "everyone assumes it" premise isn't in the review checklist, and the domain review wouldn't think to ask either. His solution was to write it as a new rule into the domain checklist — "for times shown to humans, ask which timezone first" — so the next domain review would check it. Users can also add pitfalls their own projects have hit, one by one.

Finally, the creator recommends a "magic spell": "Starting from first principles and the goal of this requirement, re-research whether there is a simpler, more suitable, and cheaper solution." This sentence makes the AI return to the goal and re-research; most of the time it optimizes the solution for another round, and sometimes it overturns and redesigns it entirely. He gives an example: adding tests to a project, the AI recommended changing all places in production code that get the current time into replaceable interfaces so that time could be frozen in tests, and this was written into the standard. Later, after asking this magic spell, the AI went back to check the code and found that changing a dozen places in production code only added tests for a one-second boundary — not worth it. In the end, not a single line of production code was changed, and the standard was relaxed — future time checks just pass the current time in as a parameter. However, he also tried putting this sentence into the SDFlow workflow, and the effect wasn't as good as asking it separately, presumably related to context and attention mechanisms.

Back to the opening question: can an AI-written Spec be used directly to generate code? The creator's answer is: run a review pass first. AI reviews from several angles simultaneously; findings first get citation verification, then item-by-item adjudication; humans don't need to watch, just read the review report and give feedback, then make the final approval. The time and tokens spent on review buy two things — less uncertainty and earlier problem detection. What the review can't catch, once discovered, gets written into the checklist as a rule and can be used next time. But spec review cannot replace a person's judgment about their own requirements; what it ultimately becomes still requires you to make the decisions.

(This article is based on the video content of Bilibili creator "Lao Dao Qian Li"; the SDFlow repository address is in the video description.)