[KongchangAI]
· 2 min read· 1,127 words

PatternBloom: Externalizing RL Distillation with Graph Patterns to Teach Agentic RAG How to Reason

PatternBloom: Externalizing RL Distillation with Graph Patterns to Teach Agentic RAG How to Reason

PatternBloom shifts Agentic RAG's retrieval unit from content to logic, externalizing procedural knowledge via two-stage RL.

PatternBloom tackles the plateau in agentic RAG where procedural knowledge is trapped in hidden states as reasoning depth grows. It shifts retrieval from "content" to "logic" using two-stage RL: Information-Density Reward selects dense evidence graphs, and high-scoring trajectories are distilled into type-abstracted reasoning skeletons in Graph Pattern Memory; Pattern-Augmented Reward then teaches the policy to match and reuse these patterns. A 7B model surpasses 27 baselines by +15.0 on Avg-OOD EM, with gains compounding as reasoning depth increases.

Agentic RAG is becoming the mainstream paradigm for retrieval-augmented generation, but its ceiling is becoming increasingly obvious: whether retrieving text passages, knowledge graph fragments, or hyperedges, reasoning paths remain trapped in the hidden states of the language model. Declarative knowledge (what to read) and procedural knowledge (how to reason) are conflated, causing retrieval methods to plateau as reasoning depth increases. PatternBloom proposes a new approach—shifting the unit of retrieval from "content" to "logic," and externalizing reasoning patterns into reusable graph structures through reinforcement learning.

PatternBloom paper page screenshot

From Content Retrieval to Logic Retrieval: What's Wrong

Existing RL-trained RAG agents typically supervise training with outcome rewards based on final answer correctness. The model learns to "assemble answers from retrieved content," but the reasoning steps repeatedly required for the same type of query—such as multi-hop comparison, causal chain tracing, and set intersection—are not explicitly modeled. This procedural knowledge must be re-derived in hidden states every time, and the deeper the reasoning, the greater the cost of redundant work.

The PatternBloom authors point out that this is the fundamental reason why content-only methods hit a plateau as reasoning depth grows. The retrieved content itself is not wrong; what's missing is an independent, reusable layer of reasoning skeletons.

The distinction between declarative and procedural knowledge originates from cognitive science: the former refers to "knowing what" (e.g., facts, concepts), and the latter to "knowing how" (e.g., reasoning steps, operation sequences). In RAG systems, retrieved text passages and graph fragments belong to declarative knowledge, while reasoning processes such as multi-hop comparison and causal tracing belong to procedural knowledge. Traditional methods mix the two in the language model's hidden states, making procedural knowledge impossible to reuse explicitly. PatternBloom's core insight is precisely to decouple the two, externalizing procedural knowledge as graph structures for storage.

Two-Stage RL: First Learn Patterns, Then Learn to Use Patterns

PatternBloom's framework consists of two reinforcement learning stages, with a pattern memory bank in between.

Stage 1: Information-Density Reward for Selecting Evidence Graphs

The agent constructs an evidence graph at query time. In the first stage, a frozen oracle computes the information gain brought by this graph, and an Information-Density Reward (IDR) is designed as a scale-normalized proxy for mutual information. The key to IDR is "density": it's not that more evidence is better, but that the information gain per unit of evidence should be high. This prevents the reward model from encouraging the piling up of retrieved content.

Mutual information measures the degree of statistical dependence between two variables; here it refers to the amount of information shared between the evidence graph and the correct answer. Directly computing mutual information is usually infeasible, so a proxy is needed. IDR adopts a "scale-normalized" mutual information proxy, dividing information gain by the size of the evidence graph (e.g., number of nodes or edges), thereby rewarding graphs with "high information per unit of evidence" rather than simply accumulating evidence. This density-based reward design effectively prevents the agent from gaming the reward by retrieving large amounts of low-quality content.

Graph Pattern Memory: Distilling High-Scoring Trajectories into Reasoning Skeletons

High-reward trajectories are not directly used as training samples; instead, they are distilled into type-abstracted reasoning skeletons and stored in Graph Pattern Memory (GPM). This step is the core of the entire method—"type abstraction" means that what is stored in memory is not the evidence content of a specific query, but the reasoning step structure of that class of logic. Procedural knowledge is thus stripped from hidden states and becomes an externalized, retrievable, and reusable structural prior.

Type abstraction refers to extracting a general structure applicable to similar problems from specific instances. For example, multiple "multi-hop comparison" queries may involve different entities and relations, but their reasoning step sequences (e.g., first find A's attribute, then B's attribute, finally compare) are the same. GPM stores exactly this step skeleton with specific content removed, enabling patterns to be reused across queries. This is analogous to function abstraction in programming: defined once, called in many places.

Stage 2: Pattern-Augmented Reward to Align Policy with Memory

The second stage uses Pattern-Augmented Reward (PAR) to couple the policy with memory. The model must not only answer correctly but also invoke matching graph patterns during reasoning. In this way, procedural reasoning shifts from "deriving on the fly every time" to "learning to match and reuse existing patterns."

Reward design in reinforcement learning directly affects the direction of policy learning. Outcome rewards focus only on final answer correctness, which can lead the model to learn "shortcuts" and ignore the reasoning process; process rewards evaluate intermediate steps. PatternBloom's two-stage design effectively combines both: the first stage uses IDR as a process reward to select high-quality evidence graphs, and the second stage uses PAR to reward both answer correctness and pattern matching, thereby guiding the policy to learn reusable reasoning paths.

Results and Significance: 7B Model Outperforms 27 Baselines

The paper's core data: the 7B-scale PatternBloom surpasses 27 baselines by +15.0 on Avg-OOD EM. A more critical observation is that this gain compounds as reasoning depth increases, because the distilled procedural knowledge is reused rather than re-derived at each query.

This result validates the authors' core argument: externalizing procedural knowledge as structural priors is an effective path to break through the content-retrieval plateau. For agentic RAG researchers, PatternBloom provides a reproducible starting point—the code is open-sourced on GitHub (EllenYiGe/PatternBloom).

Open Questions Worth Noting

PatternBloom's value lies in proposing a new level of abstraction, but it also leaves several open questions. How the number and granularity of patterns in GPM should scale with task distribution, how the boundaries of type abstraction should be defined, and whether the frozen oracle's information gain estimation remains reliable in open-domain tasks are all directions for future work. Whether the externalization of procedural knowledge can transfer across tasks also warrants further verification.

One-line Summary

PatternBloom shifts Agentic RAG's retrieval unit from content to logic, externalizing procedural knowledge through two-stage reinforcement learning; a 7B model outperforms 27 baselines.

Paragraph Summary

PatternBloom addresses the problem in agentic RAG where procedural knowledge is trapped in hidden states and performance plateaus as reasoning depth increases, proposing to shift the retrieval unit from "content" to "logic." The method comprises two RL stages: the first uses Information-Density Reward to select high-information-density evidence graphs and distills high-scoring trajectories into type-abstracted reasoning skeletons stored in Graph Pattern Memory; the second uses Pattern-Augmented Reward to teach the policy to match and reuse these patterns. Experiments show a 7B model surpasses 27 baselines by +15.0 on Avg-OOD EM, with gains compounding as reasoning depth increases. This work provides a reproducible starting point for externalizing procedural knowledge and leaves open questions such as pattern scaling and the boundaries of type abstraction.

#Agentic RAG#PatternBloom#reinforcement learning#procedural knowledge#graph pattern memory#retrieval-augmented generation
Share:

Related articles