AI Agents Don't Need Memory — They Need Documentation

A Misunderstood Premise: The Agent "Memory" Problem

In discussions about AI agents, "memory" has become the default technical goal. Whether it's vector databases, long context windows, or elaborate memory architectures, the industry seems to have converged on a consensus: to make agents smarter, you need to make them "remember" more.

But a post on Hacker News titled "Agents don't need memory, they need documentation" pushes back against this mainstream narrative. Its core argument is simple: we've been using the wrong framework to understand what agents actually need. The real gap isn't "memory" — it's "documentation."

The Fundamental Difference Between Memory and Documentation

This distinction might sound like wordplay at first, but it points to two fundamentally different engineering philosophies.

Memory Is Implicit, Volatile, and Hard to Audit

When we talk about an agent's "memory," we typically mean implicit storage: conversation history compressed into vector space, key facts stuffed into some state object. The problem with this approach is that it's a black box. It's hard to know exactly what the agent "remembers" or "forgot" at any given decision point, and even harder to trace errors back to a specific cause. Memory drifts over time, gets overwritten, and can produce hallucination-like "false memories."

Vector databases are the most common implementation of agent "memory" today: text is converted into high-dimensional vector embeddings, and relevant "memories" are retrieved via similarity search. The fundamental limitation here is the irreversibility of semantic compression — raw information inevitably loses detail during encoding, retrieval depends on distance algorithms rather than logical reasoning, and the same text can produce drastically different retrieval behavior across different embedding models. More critically, the vector space itself is unreadable to humans. Engineers can't directly inspect stored content to determine what the agent "knows," making debugging and auditing extremely difficult. This is also where "false memory" problems arise: when semantically similar but semantically inaccurate content is retrieved, the model may output incorrect "recollections" with high confidence, with no warning mechanism at the system level.

Documentation Is Explicit, Persistent, and Verifiable

Documentation follows a different logic entirely. It's knowledge that is explicitly written down, human-readable, and version-controlled. Good documentation tells the agent "how things should be done," "what the constraints of this system are," and "what the reasoning behind past decisions was." It doesn't depend on context surviving from a previous conversation — it exists as an external reference that can be consulted at any time.

In other words, memory tries to make the agent "remember everything itself," while documentation lets the agent "look up whatever it needs." The latter is much closer to how human experts actually work — no engineer memorizes an entire codebase. They rely on documentation, comments, and search.

RAG (Retrieval-Augmented Generation) is the most mainstream engineering implementation of the documentation-first approach: it combines a structured external knowledge base with a language model, allowing the model to retrieve relevant document fragments as context in real time during generation. The key difference from vector memory is that RAG's knowledge source consists of human-readable, human-editable raw documents. The retrieval process is relatively transparent, and updating knowledge simply requires editing a document — no retraining or rebuilding embedding indexes needed. Version control (like Git) can track the full evolution of documentation, with every knowledge change leaving a clear trace. This gives RAG architectures natural "auditability" — when an agent gives a wrong answer, you can trace it back to the specific document that provided the faulty information and fix it precisely, rather than blindly tuning parameters inside a black-box system.

Why Documentation May Be the More Pragmatic Path

From an engineering standpoint, betting on "documentation" over "memory" offers several practical advantages.

Greater controllability. Documentation is something humans can directly edit and review. When an agent behaves incorrectly, you can modify the relevant document rather than being helpless in the face of an opaque memory system. This ability to intervene is critical for reliability in production environments.

Better consistency. Multiple agent instances can share the same documentation as a single source of truth, avoiding the state fragmentation that comes from each instance maintaining its own separate memory.

Lower cost and complexity. Maintaining a well-structured document store with a retrieval mechanism (RAG-style solutions) is generally more predictable and easier to debug than building and tuning complex long-term memory architectures.

This Doesn't Mean Memory Has No Value

To be clear, this argument doesn't wholesale reject the value of memory — it's about reprioritizing. Short-term working memory — such as maintaining intermediate state during a task execution — is still necessary. What's being questioned is the tendency to use "anthropomorphized long-term memory" as a substitute for solid knowledge engineering.

When teams invest heavily in making agents "remember what a user said last week," they may be overlooking a more fundamental question: does the agent have clear, accurate, structured documentation to guide it through its tasks? In many failed agent applications, the root cause isn't "it forgot" — it's "it was never properly informed to begin with."

The concept of "working memory" is borrowed from cognitive science: when humans perform complex tasks, they temporarily hold a small number of highly relevant pieces of information in mind (such as intermediate calculation results or the current focus of a conversation), and these are not stored long-term after the task ends. In agent engineering, working memory typically refers to the contents of the context window during a single task execution — including current instructions, tool call history, and intermediate outputs. Modern large language models now support context windows of hundreds of thousands of tokens, which is generally sufficient to maintain necessary state throughout a complete task flow. What's truly contested is "cross-session long-term memory": attempts to have agents accumulate personalized memory across different tasks and users. This kind of need has legitimate use cases in consumer assistant products, but in enterprise-grade tool-oriented agents, the added state management complexity far outweighs the practical benefit.

Implications for Builders

For teams building agent products, this perspective offers a practical path worth considering:

• Before piling on complex memory systems, confirm whether the knowledge required for the task has already been explicitly documented.
• Treat "documentation quality" as a first-class citizen of agent capability, not an afterthought.
• Replace some of what you'd hand off to "memory" with a retrievable, version-controlled external knowledge base.
• Keep lightweight working memory for in-session state, but don't make it carry the burden of knowledge storage.

Conclusion

The Hacker News post itself didn't generate much discussion (only a handful of votes, no comments at the time of writing), but the premise it raises is sharp: while the entire industry is chasing "agents with memory," the real bottleneck may be that the documentation we give agents is simply too poor. Think of an agent as a smart new hire — it doesn't need you to implant memories. What it needs is a clear, well-organized manual it can get up to speed with and reference at any time. That simple analogy may cut closer to the heart of the problem than many elaborate memory architectures.