RAG Meets Agent Memory: How Agentic GraphRAG Keeps Agents from Forgetting

Anyone building AI agents has probably run into this awkward situation: the agent made a decision yesterday based on some policy document, and today the document has been updated—but the agent has no idea and keeps following the old rules. The problem isn't that the model isn't smart enough; it's that the context layer (RAG) and the memory layer (Agent Memory) have long been managed separately. An author who has spent three and a half years in RAG, LLM training, and retrieval—and who contributed to the MMTEB paper—shared on Reddit the Agentic GraphRAG project they're building, an attempt to stitch these two layers together using a single unified graph.

Two Disconnected Layers: An Agent's "Knowledge" and "Experience" Don't Talk to Each Other

In today's agent ecosystem, two camps work independently. One camp builds the Context Layer: hybrid RAG, GraphRAG, wiring retrieval pipelines into agents so models can look up documents and query graphs. The other camp builds the Memory Layer: letting agents store what they've done and the decisions they've made, so they can pick up where they left off next time—coding agents are furthest ahead in this regard.

The pain point the original author observed is this: these two systems lack real integration. An agent makes a decision based on some policy document and records it, then the document changes—but no mechanism tells the agent "that decision of yours is now outdated." Memory is memory, knowledge is knowledge, and there's no thread connecting them. Agentic GraphRAG is precisely aimed at solving this "decision expires with no one to remind you" problem.

RAG (Retrieval-Augmented Generation) is a technical paradigm in which a large language model retrieves external knowledge bases before generating a response. It typically involves document chunking, vector embedding, similarity retrieval, and context assembly, enabling the model to cite up-to-date information beyond its training data. Agent Memory, on the other hand, focuses on the persistent storage and reuse of an agent's own experiences, task states, and historical decisions during interactions, commonly taking the form of short-term conversation buffers, long-term vector memory, and structured event logs. Although both involve "letting the model know more," RAG emphasizes retrieval of external static knowledge while Memory emphasizes accumulation of internal dynamic experience; the former updates through document re-indexing, the latter through event writes. Due to the lack of a unified identification and linking mechanism, document version changes cannot automatically propagate to decision records in memory—this is precisely the root of the disconnect described in the article.

Three-Layer Graph Structure: Knowledge, Working Memory, and Decision Memory

The project's core design is to store data in a single graph, divided into three interconnected layers.

The knowledge layer plays the role of the Context Layer, storing documents, source passages, entities, and their relationships. Ingestion uses Docling, chunking uses Chonkie, and chunking preserves the document's section structure, tables, and paragraph hierarchy. Each passage carries source version and location information, and PDFs also record page number positions—meaning citations can be precise down to "which page, which paragraph."

The working memory layer stores tasks, progress, messages, and tool results. The built-in agent is based on DeepAgents/LangGraph, recording messages and tool calls as task events, so subsequent sessions can resume task progress rather than starting from scratch each time.

The decision memory layer stores conclusions, assumptions, and open questions, each linked to the exact source evidence, recorded events, or earlier decisions. Decisions can be reaffirmed or replaced, and history is never lost.

The three layers are connected through the graph, and this is where the value of the "unified graph" lies: decisions can be traced to evidence, evidence can be traced to document versions, and when a document changes, affected decisions can be identified.

GraphRAG is a graph-structured variant of RAG: rather than being content with cutting documents into isolated text chunks for vector retrieval, it first extracts entities and relationships to build a knowledge graph, then combines graph traversal and community summarization to enhance retrieval. Compared to traditional vector RAG, GraphRAG's advantage lies in answering questions that require multi-hop reasoning, such as "which departments were affected by a certain policy, and what projects did those departments execute?" Neo4j is a mainstream native graph database that supports the property graph model, the Cypher query language, and built-in full-text and vector indexes, making it a common storage and retrieval foundation for GraphRAG. Placing the knowledge layer, working memory layer, and decision memory layer in the same Neo4j graph means all three node types share the same identification and relationship types, and decision nodes can point directly via edges to evidence passages and document version nodes—enabling cross-layer traceability and invalidation propagation.

Engineering Details of the Knowledge Pipeline

The knowledge pipeline is built around user-defined schemas, covering entity types, attributes, and relationships. The schema both guides extraction and validation and provides entity labels for graph search. Extraction can run GLiNER locally, and selected chunks can also use BAML for structured LLM extraction. The author specifically emphasizes: even if extraction misses a fact, searching directly by passage still works—this is a fallback for extraction failures.

For retrieval, passages are found through Neo4j's native full-text and vector indexes, with reciprocal rank fusion combining keyword rankings and semantic rankings. Even more practical is the ability to "read beyond the search hit": you can navigate to the section containing the hit passage, adjacent passages, or table rows with headers; entity relationships can also point back to the supporting passages that record evidence. The entire design philosophy is to make retrieval results extensible and traceable, rather than an isolated piece of text.

Docling is IBM's open-source document parsing tool, adept at converting PDF, DOCX, and other formats into structured Markdown or JSON, preserving heading hierarchy, tables, lists, and reading order, and outputting location information such as page numbers. Chonkie is a lightweight text chunking library that supports splitting by semantics, sentences, or fixed windows, and can align with document structure. GLiNER is a zero-shot named entity recognition model that can extract entities according to user-specified entity types without domain-specific fine-tuning, making it suitable for localized, low-cost entity extraction. BAML is a domain-specific language for structured LLM output, allowing you to define extraction schemas and generate type-safe calling code. Reciprocal Rank Fusion (RRF) is an algorithm that fuses multiple retrieval rankings without training: it takes the reciprocal of each document's rank in each ranking list and sums them to obtain a fused score, effectively balancing the ranking differences between keyword retrieval and semantic retrieval.

What the Agent Can Actually Do

These connections translate into several concrete capabilities for the agent:

• Cite the exact source version. The built-in agent checks whether each cited passage was actually retrieved during the current run, and whether it belongs to a document the agent has permission to read—this amounts to adding a validation layer to citations.
• Retrieve earlier decisions within the same project.
• When a cited document changes, affected decisions and their explicit dependencies are flagged for review. This is the direct solution to the "decision expiration" problem from the opening.

Roadmap and Open Questions

The author lists a three-item roadmap: make the pipeline more modular so components can run on different hardware; extend retrieval support to more graph databases and storage backends; and run large-scale benchmarks on domain-specific datasets, with results to be released soon.

The question the post ultimately throws to the community is: what tools do you use for RAG and memory in your agents? The question itself shows that there is currently no universally accepted standard answer for unifying context and memory. Agentic GraphRAG's approach is to "use a graph to string together the evidence chain and the decision chain"—whether it holds up under real workloads remains to be seen from the forthcoming benchmark data.

For teams building agent systems, what's worth paying attention to in this project isn't whether it uses Neo4j or Docling, but that it raises a previously overlooked engineering requirement: a memory system must know when its own basis becomes invalid. Without this, the longer an agent's memory persists, the more outdated decisions it accumulates.

One-line Summary

Agentic GraphRAG uses a unified graph structure to connect RAG context with Agent memory, solving the problem of old decisions going unflagged after documents are updated.

Paragraph Summary

The article introduces an open-source project called Agentic GraphRAG, aimed at solving the current problem in AI agents where the context layer (RAG) and memory layer (Agent Memory) operate independently and lack integration. The core design stores knowledge, working memory, and decision memory as three layers in the same graph, with decision nodes linked to exact source evidence and document versions, so that when documents change, affected old decisions are automatically flagged. The knowledge pipeline uses Docling for ingestion, Chonkie for chunking while preserving document hierarchy, and retrieval combining Neo4j full-text and vector indexes with reciprocal rank fusion for ranking. The built-in agent validates whether cited passages were actually retrieved in the current run and belong to readable documents. The project roadmap includes pipeline modularization, expanding to more graph database backends, and forthcoming benchmarks on domain-specific datasets.