Should OpenWiki Index Narrative Documents Like PRDs? A Trade-off Analysis

Background

A developer posted a seemingly simple but surprisingly common question on Reddit: when building an OpenWiki (an automated code knowledge base / documentation system) for a project, should you include narrative documents — PRDs (Product Requirements Documents), design docs, strategy memos, and vision statements — in the index, or should the wiki only be generated from the code itself?

The core dilemma was this: should you exclude the docs folder and feed only code, or include the documents alongside the code — and which approach actually makes the wiki better or worse? This question cuts to a fundamental design trade-off in automated documentation tools: the balance between breadth of context and signal-to-noise ratio.

The core mechanism of OpenWiki-style tools typically works like this: the tool scans the codebase via static analysis or an LLM, automatically extracting function signatures, module relationships, comments, and other structured information to build a searchable knowledge base. Users can ask questions in natural language, and the tool retrieves relevant snippets via vector search (Embedding + semantic similarity) to generate answers. The retrieval quality in this architecture depends heavily on the consistency and accuracy of indexed content — if the index contains contradictory information, vector search cannot automatically determine which content is more trustworthy, and will include both in the generation context, producing confused answers. This is the core technical reason why "what to put in the index" is such a critical question in RAG (Retrieval-Augmented Generation) systems.

The Fundamental Difference Between the Two Document Types

To answer this question, we first need to understand the nature of code versus narrative documents.

Code: Precise but Lacking Intent

Code is a precise description of how the system works. A wiki generated from code can accurately reflect function calls, module dependencies, data structures, and other factual information. But code alone often can't answer why something was designed this way — intent, trade-offs, and historical decisions are typically buried outside the code itself.

Narrative Documents: Rich in Intent but Prone to Staleness

PRDs, design documents, and vision statements carry exactly what code is missing: product goals, design rationale, and strategic direction. They help wiki readers understand the why, which is especially valuable for onboarding new team members or enabling cross-team collaboration.

But narrative documents have a natural weakness — they tend to drift out of sync with the actual code. A PRD written six months ago may describe features that were later cut; a design document may document an architecture that has since been refactored. If the wiki presents this content as fact, it can actively mislead.

This kind of "Semantic Drift" between code and documentation is extremely common in software engineering. Research shows that documentation is typically maintained far less frequently than code changes, especially in fast-moving teams. PRDs are often written when a feature is first scoped, and as implementation decisions evolve during development, the document doesn't always get updated. Before automated wiki tools existed, this drift had relatively limited impact — engineers would use experience to judge how current a document was. But when documents are processed by an LLM as equally-weighted "sources of truth," their staleness gets amplified into concrete wrong answers, making the problem far more harmful.

Benefits and Risks of Indexing Narrative Documents

Given the nature of this trade-off, the key considerations break down clearly on both sides.

Potential Benefits

• Filling in the "why" dimension: Code tells you how; narrative documents tell you why. Combining both makes the knowledge base significantly more complete.
• Improving retrieval recall: When users ask questions in natural language (e.g., "why did we choose this approach?"), narrative documents are often the only content source that will surface a relevant answer.
• Helping new members understand the big picture: Vision and strategy documents help readers understand the overall direction of the project, not just individual implementation details.

Potential Risks

• Outdated information creates misleading answers: Stale PRDs or design drafts may cause the wiki to return answers that don't match the current state of the system.
• Introducing noise: Drafts, brainstorming notes, and rejected proposals, if mixed into the index, will dilute high-quality content.
• Factual conflicts: When document descriptions contradict code implementations, the auto-generated wiki may be unable to determine which one is true.

Practical Decision-Making Recommendations

Rather than choosing between "index everything" and "index nothing," a more robust approach is to apply layered processing.

Filter by Document Maturity

Don't blindly throw the entire docs folder into the index. Prioritize documents that are relatively stable and finalized (e.g., formal architecture designs, PRDs for already-shipped features), and exclude obvious drafts, temporary notes, and abandoned proposals.

Annotate Source and Recency

If the wiki tool supports metadata, tag narrative documents with labels like "document type" and "last updated date." Even if the content makes it into the index, readers can then make their own judgment about its credibility, reducing the risk of being misled by stale information.

Keep Code as the "Source of Truth"

When information conflicts, establish a clear principle: code is the ultimate source of truth; narrative documents are a supplement to intent. Some wiki tools allow you to set retrieval weights for different content sources — use that to give code-based content higher priority in retrieval.

Validate Incrementally

The lowest-cost approach is to generate two versions — one code-only and one including documents — then test retrieval quality with a handful of representative questions, and decide whether to keep the documents based on actual results. This is more persuasive than any theoretical debate.

Conclusion

Whether OpenWiki should index narrative documents is not an absolute yes or no. Narrative documents can significantly enhance the knowledge base's ability to answer why questions — but only if you invest in proper filtering, annotation, and conflict resolution.

The recommended path is: include mature, finalized narrative documents; exclude drafts and abandoned proposals; use code as the factual baseline; and validate the actual impact through a small-scale A/B test comparing the code-only and document-inclusive versions. This lets you capture the contextual benefits of narrative documents while keeping the risk of stale information within an acceptable range.

Supplementary Notes

In RAG systems, assigning differential weights to documents from different sources is typically achieved in two ways: first, adjusting the score weights of documents from each source during the retrieval phase (Reranking) so that code snippets rank higher in the candidate list; second, explicitly instructing the model via the System Prompt during the generation phase that "if code and documentation conflict, defer to the code." Both approaches can be used together. Some tools (such as Cursor and Mintlify) have started supporting index priority configuration by file type or directory. For tools that don't yet support this, you can manually add disclaimer-style prefix annotations to narrative documents during the preprocessing stage, which indirectly influences the model's degree of trust toward that content.