Is Your AI-Written README Lying? An Evidence-First Documentation Workflow

Ask Cursor, Claude Code, or ChatGPT to "write a README for this repo" and you'll get something that looks exactly like a README: feature list, installation steps, quick start, usage examples. Until a new colleague follows the quick start and fails at step two—the script is called dev, not start; the --verbose flag in the docs never existed; the .env.example it told you to copy isn't even in the repo.

The model isn't broken. It did exactly what was asked: produce complete documentation. When information gaps appeared, it filled them with "what projects like this usually have." Fluent prose makes guesses look identical to facts.

Why "Write Docs for This" Goes Wrong

The failure modes are predictable:

• Convention filling: npm start, --config, DATABASE_URL, LOG_LEVEL—each one sounds plausible, but any of them could be wrong.
• Fabricated reasons: Ask for the "why" and you'll get explanations like "to avoid rate limiting" or "for performance" that nobody ever actually gave. The next maintainer will believe them.
• Untested commands: Installation steps nobody ran look exactly like the ones someone did.
• Confident numbers: Defaults, limits, timeouts, supported versions—the most-copied parts of documentation are also the easiest to fabricate.
• Write-once, drift-forever: Even a perfect doc starts drifting at the next merge.

These problems don't need a smarter model. They need a workflow where the model must show its sources and where "I don't know" has somewhere to go.

Hallucination is an inherent property of large language models: they predict the next most likely token from context rather than querying a database. When a pattern appears repeatedly in training data (like the npm start common in READMEs), the model completes it with high confidence regardless of whether the current repo actually matches. This has little to do with "intelligence"—even GPT-4-class models can't fundamentally eliminate it.

That's why the article emphasizes that the solution isn't "use a stronger model" but changing the task structure: turn open-ended writing into an evidence-constrained fill-in-the-blank exercise. Once the model is required to attach file:line to every fact, the prediction space collapses from "what projects like this usually have" to "what this repo actually has," and hallucinations drop dramatically. This aligns with the RAG (Retrieval-Augmented Generation) approach, but at finer granularity—down to the line-number level.

A related concept is model calibration: ideally, a model should express low confidence about uncertain content. But conversational models trained with RLHF are often optimized to "give a complete answer," making fluent text and factual text indistinguishable on the surface—which is exactly why the README at the start of this article "looks exactly like a README."

The Core Workflow: Evidence → Outline → Draft → Verify → Sync

1. Evidence: Before writing a single word, clarify who the reader is and which facts the docs depend on, with each fact carrying a source (file:line, commit, PR, notes). Anything evidence doesn't cover becomes a question.
2. Outline: Map sections to evidence lines. Cut sections with no evidence outright or keep them as visible TODO placeholders. Cutting a section at this stage costs nothing.
3. Draft: Text constrained by the outline. Commands copied from scripts, not "improved." Unknown content written as TODO: <question>.
4. Verify: Run the copy-paste path on a clean machine. Grep every identifier. Check every number and every "because."
5. Stay in sync: Audit docs against code before each release, update runbooks after each incident.

If you take away one rule from this article: a visible TODO beats a plausible guess. Tell the model this explicitly. Models are trained to be "helpful," and the default helpful behavior is filling in blanks. Tell it that a blank is the helpful answer, and your results will change.

The "visible TODO" here isn't about laziness—it's an explicit knowledge-boundary marker. Traditional docs are dangerous because they mix "known" and "guessed" in the same paragraph, leaving readers unable to distinguish them. A TODO placeholder forces the two apart: when readers see TODO, they know they need to confirm with a maintainer; when they see prose, they can trust it.

This principle aligns with the fail-fast philosophy in software engineering: rather than letting errors propagate silently downstream (a new colleague following steps and failing), expose the gap at the source. In practice, a TODO should also carry a specific question (TODO: is this parameter's default 30 seconds or 3 minutes?) rather than a vague TODO: complete, because only answerable questions actually get answered.

For the model, explicitly telling it "a blank is the helpful answer" is a form of inverse incentive. Conversational models are trained by default to fill blanks in order to seem useful—only by redefining "useful" explicitly in the prompt does the model's behavior shift. This is the shared underlying logic of all the prompt rules here: use rules to counter training tendencies.

Free Prompt One: Write a README Based on Evidence, Not Vibes

This prompt works with Cursor (use @ to reference the repo), Claude Code (can read files), or ChatGPT/Copilot Chat (paste the file tree, manifest, and scripts).

You are a senior engineer writing the README for this repository. You write only what the code, config, and files I give you can support. Anything you cannot support from evidence becomes a visible TODO, not a guess.

## Context
- Repo/paths to read: {{REPO_PATHS}}
- Project purpose, in my words (may be incomplete): {{PROJECT_PURPOSE}}
- Who reads this README: {{AUDIENCE}}
- Commands I know work, or "unknown": {{KNOWN_COMMANDS}}

## Process (plan before you write)
1. Evidence scan. Read manifest files, scripts (Makefile, package scripts, CI workflows), entry points, config and env var reads, LICENSE, and existing docs. Record every fact you rely on and its source file:line.
2. Gaps. List what readers need that the evidence doesn't tell you. Each gap becomes a TODO with a question for the maintainer.
3. Outline. Propose sections; annotate each with the evidence lines it uses. Stop here and wait for my approval.
4. Draft. Quick start in clean-clone order: prerequisites, install, configure, run, test. Commands copied verbatim from scripts.
5. Verification checklist. Every command, path, env var, and flag in the README, with sources, marked RAN (only if you actually executed it) or UNVERIFIED.

## Output
### Evidence Table | # | Fact | Source (file:line) | Section used |
### Gaps & TODOs
### Proposed Outline
### README.md (after approval)
### Verification Checklist | Item | Source | RAN / UNVERIFIED |

## Rules
- Never invent APIs, CLI flags, env vars, config keys, commands, defaults, or behavior that don't exist in the code. Mark unknowns as TODO: <question>.
- Don't add badges, registries, or CI services the repo doesn't configure.
- Don't use phrases like "blazing fast" or "production-ready" without evidence.
- The license line must match the LICENSE file; if there isn't one, write TODO.
- Short and correct beats long and plausible.

Three parts do the heavy lifting:

• Evidence table: Every section must point to evidence lines. A Docker section with no Dockerfile has nothing to point to.
• Pause after outline: You review a ten-line structure, not three pages of prose.
• RAN / UNVERIFIED: In a chat interface, everything is UNVERIFIED, which is honest. You run the quick start, then flip the tags.

Expect some TODOs. A README written from incomplete material with zero TODOs is the one to be suspicious of.

Free Prompt Two: Documentation Drift Audit

The docs used to be correct. But then a flag got renamed, a default changed, a new required env var appeared. This prompt turns "the docs look fine" into an actionable ledger.

You are a meticulous technical reviewer auditing documentation against the current code. You treat every specific statement in the docs as a claim to be verified. Code is the source of truth; you never modify code to make the docs correct.


**Documentation Drift** is the phenomenon where docs and implementation gradually diverge as code evolves. It shares roots with code rot but is harder to notice because docs don't fail to compile. Drift goes in two directions: docs describe old behavior that no longer exists (forward drift), and code adds user-facing capabilities the docs never mention (backward drift). Most teams only check the former, but the UNDOCUMENTED verdict in prompt two exists precisely to cover the latter—these gaps are often more dangerous because users don't even know the new feature exists.

The granularity of the audit matters too: breaking "every specific statement in the docs" into verifiable atomic claims (commands, flags, defaults, ports, limits) is the prerequisite for making the audit mechanical. A vague "the intro is unclear" can't be verified, but "is `--timeout`'s default 30 or 60" can be adjudicated directly with `grep`.

Finally, the rule that **code is the source of truth** matters especially in agent scenarios. A write-capable agent told to "make the docs correct" might reverse-modify code or reintroduce deleted flags to match the docs. Explicitly writing "docs only, never touch code" is a key guardrail against automated tools doing harm with good intentions.
## Context
- Docs to audit: {{DOC_PATHS}}
- Code to verify against: {{CODE_PATHS}}
- What changed since the docs were last correct (git log / diff --stat, or "unknown"): {{CHANGE_SUMMARY}}
- Who reads these docs and what they do with them: {{READER}}

## Process
1. Extract every checkable claim and give it an ID: commands, CLI flags, env vars, config keys, defaults, endpoints, file paths, function names, versions, ports, limits, behavioral statements ("retries three times").
2. Verify each against the code:
   MATCH (cite line) / DRIFT (code says X, docs say Y; cite both) / MISSING-IN-CODE (list search terms) / UNDOCUMENTED (user-facing in code but not mentioned in docs) / UNVERIFIABLE (state what would settle it).
3. Rank by reader harm: HIGH = copy-paste fails, data loss, security, wrong limits; MEDIUM = edge-case errors; LOW = wording. Attach a confidence level to each verdict.
4. Report and stop. Don't edit anything.

## Output
### Summary (counts per verdict, top five HIGH-harm items)
### Claim Ledger | ID | Doc location | Claim | Verdict | Evidence/search terms | Harm | Confidence |
### Undocumented User-Facing Changes
### Prevention (1-3 suggestions using tools the repo already has)

## Rules
- Never invent APIs, flags, env vars, or behavior.
- If the docs describe behavior the code doesn't have, report DRIFT and ask which is correct.
- UNVERIFIABLE is an acceptable verdict. Don't upgrade it to MATCH.
- Mark uncertain items with TODO: verify <question>.

Two details matter:

• Check both directions: Outdated claims in the docs are only half of drift; the other half is new required env vars nobody wrote down—that's the UNDOCUMENTED section.
• "Code is the source of truth" is a rule, not a feeling: Without it, a write-capable agent might "helpfully" re-add deleted flags to make the README correct again.

Run this on the docs users hit first before release: README quick start, install page, configuration reference. Paste the release's git diff --stat as the change summary so the model knows where drift is most likely.

Why Comments: Rules That Save the Next Maintainer

Comments that narrate code (# loop through users) are noise. Comments that explain "why" are gold—and also where models fabricate the most. The fix is a simple constraint:

Why-comments must come from the code itself, a ticket, a commit message, or a person. Otherwise write TODO(why): <question>.

Then read the list of TODO(why) questions the model produces. This is often the single most valuable output of the entire session: a precise list of knowledge that will vanish when someone leaves. Paste it into the PR and have authors answer.

The 10-Minute Hallucination Check

Before merging any AI-written docs:

1. Extract every command, flag, env var, config key, endpoint, and file path.
2. Search the repo for each one (git grep -n "<identifier>"). Whatever you don't find is guilty until proven innocent.
3. Run the copy-paste path in a clean environment, in order.
4. Every number (default, limit, port, version) needs a source line.
5. Every "because" needs a source. No source, no because.

Never let AI write these alone: security or compliance statements, SLAs, license text, incident disclosures, or destructive operation steps. Every sentence there is on a human.

The Structure That Works

Section	Purpose
Role	"Senior engineer writing from evidence," "scribe," "auditor"
Inputs	Paths, audience, real source material (code, PR body, notes)
Process	Evidence and outline first, text second
Output contract	Tables with Source and RAN / UNVERIFIED columns
Rules	Never fabricate; TODO for unknowns; docs only; code is source of truth

A column headed "Source (file:line)" must be filled row by row, and blank cells expose themselves in a way a confident paragraph never can. That's the whole trick.

Try it on docs you know well: regenerate your own README from code alone with prompt one, or audit it with prompt two, and check every claim and TODO it surfaces.

One-line Summary

Use evidence tables, TODO placeholders, and claim ledgers to turn AI docs from fluent guesses into a verifiable engineering process.

Paragraph Summary

The most dangerous failure mode of AI-generated docs isn't obvious errors—it's fluent, plausible guesses: a fabricated npm start, a --verbose that never existed, invented performance justifications. The problem isn't that the model isn't smart enough; it's that the task structure allows it to fill blanks. Hence the evidence-first five-step workflow: Evidence → Outline → Draft → Verify → Sync, with the core principle "a visible TODO beats a plausible guess." Two prompts accompany it: the first requires the model to output an evidence table, gap list, and RAN/UNVERIFIED verification checklist, pausing at the outline stage for approval; the second turns a docs audit into a claim ledger, checking both outdated claims and undocumented changes, ranked by reader harm. Additionally, why-comments must come from code, tickets, or people—otherwise mark TODO(why), and that list is often the most valuable output of the session.