Context as Infrastructure: The Knowledge Compilation Layer of Agentic RAG

Retrieval-augmented generation (RAG) is becoming "agentic"—search is no longer a one-shot vector retrieval but rather a process in which an agent writes queries, reads results, reformulates, and decides for itself when to stop. This "search harness" has largely solved the accuracy problem: with only BM25/grep and MiniMax M3, and without a compiled index or vector index, one run reached 85% on BrowseComp-Plus and 91% on multi-hop FRAMES. But it also exposes a subtler problem: context. An undisciplined loop will repeat queries, re-read documents, and fill the window with unused passages. The missing layer is knowledge compilation.

The search harness solves accuracy but creates a context problem

Traditional RAG centers on "retrieve—concatenate—generate," whereas agentic RAG turns retrieval into a stateful exploration process. The search harness writes queries, reads results, rewrites queries, and then decides whether to continue. The original author notes that this mechanism is already good enough on accuracy: a harness equipped with only BM25/grep and MiniMax M3 answers 85% correctly on BrowseComp-Plus and reaches 91% on multi-hop FRAMES, with no need for a compiled or vector index.

The problem lies in the "context economy." An undisciplined loop will repeatedly query the same content, re-read documents, and stuff large numbers of unused passages into the context window. Accuracy goes up, but token cost, latency, and context pollution follow. In other words, agentic retrieval has turned "finding the right thing" into a relatively easy problem, while leaving "feeding it cheaply and correctly" as a gap.

BM25 is a classic sparse retrieval algorithm that scores query-document relevance based on term frequency (TF), inverse document frequency (IDF), and document length normalization. It works without training and is commonly used in keyword-matching scenarios. grep is a text search tool that matches line by line by regex or literal pattern, suitable for quickly locating content in unindexed raw text. What they share is that they require no pre-built vector index or compiled index and operate directly on raw corpora, so they are often used as lightweight baselines for agent search harnesses. In agentic RAG, an agent can repeatedly call such tools and rewrite queries based on the returned results, rather than relying on one-shot vector similarity retrieval. The original phrase "with only BM25/grep and MiniMax M3" refers to this minimalist configuration, showing that the accuracy improvement comes mainly from the agent's query planning and iteration rather than the complexity of the retriever itself.

Knowledge compilation: not a better retriever, but a supply layer

The author's proposed solution is knowledge compilation: a configurable ingestion pipeline in which an LLM supplements documents with context that no single chunk can carry on its own—navigation hierarchies with "claim–evidence" leaf nodes, entity graphs, timelines, mind maps, ontologies, wikis, and memory.

There is an important clarification here: the compiled artifacts are not better retrievers. The original text explicitly states that, under a competent harness, structured artifacts show no detectable accuracy gain on FRAMES; PageIndex-style outlines even degrade as scale grows. The value of the compilation layer is not "searching more accurately" but supply: handing context to the model in a compiled, governable form rather than as a pile of ranked passages.

This distinction is critical. It means the metrics for evaluating knowledge compilation should not be accuracy alone, but should also include token consumption, latency, governability, and integration cost. The navigation-tree mechanism given in the original text—with leaf nodes as "claim–evidence" pairs—cuts query tokens by 91% when evidence is sufficient, increases them by 1.9% when evidence is insufficient, and reduces them by 38% on average. Based on these figures, the break-even sufficiency rate is about 2%, while the measured sufficiency rate is close to 43%. This is a fairly wide margin of profitability, suggesting that compiled artifacts are very likely worthwhile under real workloads.

PageIndex-style outlines refer to an index structure that organizes documents into a hierarchical table of contents (such as a chapter tree), similar to a book's contents page, used for navigation and locating content. Its advantages are clear structure and strong interpretability, but the original text points out that as document scale grows, such outlines degrade—excessively deep hierarchies or too many nodes lead to reduced retrieval efficiency, context inflation, and even lower accuracy than flat retrieval. This echoes the argument that "compiled artifacts are not better retrievers": structured outlines may not outperform simple retrieval as scale expands, and their value must be assessed in the context of the supply scenario.

Decision gates and the latency trade-off

Compilation is not free. The original text mentions a decision gate that cuts latency from 140 seconds to 30 seconds, but the tokens saved by the alternative mechanism are traded for retrieval rounds, which themselves carry latency cost. There is a clear trade-off structure here: the navigation tree compresses "the context that gets fed in," while the decision gate compresses "unnecessary exploration"; the two target different parts of the loop.

This also reminds us that optimizing agentic systems cannot rely on a single metric. Saving tokens may increase rounds, and reducing rounds may sacrifice recall; ultimately, a balance must be found on a multi-objective function. The "break-even sufficiency rate" mentioned in the original text is essentially asking: at what evidence sufficiency rate can the cost of compiled artifacts be covered by the benefits? The measured sufficiency rate is close to 43%, far above the 2% break-even point, indicating that this path holds statistically.

Break-even sufficiency rate is a derived metric for measuring the economics of knowledge compilation: when evidence is sufficient, compiled artifacts (such as a navigation tree) can significantly cut query tokens; when evidence is insufficient, they may instead increase token consumption. Let s be the savings ratio when sufficient and i be the increase ratio when insufficient; then average savings = s × p - i × (1-p), where p is the sufficiency rate. Setting average savings to zero yields the break-even sufficiency rate p* = i / (s + i). In the original text, s=91% and i=1.9%, giving p* ≈ 2%. The measured sufficiency rate is close to 43%, far above the break-even point, meaning that under real workloads the token benefits of compiled artifacts are very likely to cover the cost. This metric provides a quantitative basis for assessing whether the compilation layer is worthwhile.

The value of other compiled artifacts lies on the deployment side

Artifacts such as ontologies, graphs, wikis, timelines, mind maps, and memory show no advantage on accuracy benchmarks. The original text's judgment is that their returns appear at the deployment stage: governance, integration, onboarding, temporal orientation, and continuity. These are precisely the areas that benchmarks cannot see.

This observation has direct implications for engineering teams. If your evaluation looks only at accuracy metrics such as FRAMES or BrowseComp-Plus, you may conclude that "knowledge compilation is useless"; but the pain points of real systems lie elsewhere—how new members understand the data, how permissions are governed, how systems are integrated, and how conversations maintain continuity. These are exactly the areas where compiled artifacts excel.

RAGFlow's five-station pipeline and the agent-native database

The original text describes RAGFlow's pipeline and harness (open-source address: https://github.com/infiniflow/ragflow), which breaks the agentic loop into five disciplined stations, with one core goal: to ensure that the context entering the model is always the minimal sufficient set.

The author calls this combination an "agent-native database": a storage layer that hands compiled, governable context to the agent rather than returning a ranked list of passages. This positioning is noteworthy—it redefines RAG from a "retrieval pipeline" into "context infrastructure," where retrieval is only one part and compilation, governance, and supply are where the long-term value lies.

For teams building agentic systems, this means the architectural center of gravity needs to move earlier: rather than continuously tuning retrievers, first think clearly about how context is compiled, how it is governed, and how it enters the model as a minimal sufficient set. The ceiling on accuracy is often not in the retriever but in the design of the supply layer.

One-line Summary

Knowledge compilation is not a better retriever but infrastructure that supplies context as a minimal sufficient set.

Paragraph Summary

The article explores how the search harness in agentic RAG improves accuracy while creating context inflation, and proposes knowledge compilation as a supply-layer solution. The search harness alone, using only BM25/grep and MiniMax M3, can reach 85% on BrowseComp-Plus and 91% on multi-hop FRAMES, so accuracy is no longer the bottleneck; but an undisciplined loop repeats queries and fills the window with unused passages, driving up token cost and latency. Knowledge compilation uses an LLM to supplement documents with structured artifacts such as navigation hierarchies, entity graphs, and timelines; its value is not in improving retrieval accuracy (structured artifacts show no gain on FRAMES) but in supplying context in a compiled, governable form. The claim–evidence navigation tree cuts tokens by 91% when evidence is sufficient, by 38% on average, with a break-even sufficiency rate of about 2% versus a measured rate close to 43%, so the economics hold. Decision gates cut latency from 140 seconds to 30 seconds, but there is a trade-off between saving tokens and saving rounds. Artifacts such as ontologies, graphs, and wikis show their value on the deployment side—governance, integration, continuity. RAGFlow's five-station pipeline breaks the agentic loop into disciplined stations, with the core aim of ensuring that the context entering the model is always the minimal sufficient set. The article ultimately redefines RAG as context infrastructure, emphasizing that the architectural center of gravity should move from retriever tuning to supply-layer design.