Why Do AI Agents "Forget"? How a Local-First Memory Layer Fights Context Compaction

AI agents that run long enough always hit the same awkward problem: they "forget" what you told them earlier. A widely circulated article this week called context-window compaction a "200k-token lobotomy" and proposed patching it with an external state file named after the film Memento. The metaphor is vivid and honestly captures the severity of the problem. But what's really worth digging into is the design reality behind compaction — and why a plain "state file" can't actually fix it.

Compaction Isn't a Bug — It's the Design Working as Intended

The core issue is this: memory that exists only inside the context window is inherently forgetful. Compaction isn't a flaw in the design; it's the design operating exactly to spec.

The window has a token budget. When that budget runs out, something has to be evicted — and what gets cut is decided by a summariser, not by you. That means what the agent remembers and what it discards is outside your control.

A common misconception is that loading a state file into the prompt solves the problem. But as long as content lives inside the window, it will be summarised along with everything else the next time compaction runs. In other words, a state file loaded into the prompt still can't escape the "lobotomy."

The context window is the total amount of text a large language model can "see" during a single inference pass, measured in tokens. One token is roughly 0.75 English words. Current mainstream models support windows ranging from tens of thousands to over a million tokens — but no matter how large, the limit is finite. As an agent runs longer, conversation history, tool-call logs, and intermediate reasoning accumulate until they hit that ceiling.

At that point the system must choose: truncate old content, or compress it with a summarisation model and stuff the summary back into the window. The latter is context-window compaction. The problem is that summarisation is lossy by nature — details, exact phrasing, and conditional logic often vanish quietly in the process, with almost no transparency to the user. That's why the lobotomy metaphor fits: the discarded information triggers no error, it simply disappears from the model's field of view.

Five Rules: Success or Failure Is Decided at Write Time

HyperMarrow is a local-first memory layer for Windows agents. Its author argues that all of the failures described above are determined at write time, not at read time. The five rules below are ordered by when they take effect.

1. Write Decisions Down with Their Source and Timestamp

A preference, a decision, or a conclusion should land in persistent local storage as a real record — carrying where it came from and when it was made. If an assumption has to be re-derived from the conversation log every single round, it has no address, no origin, no history. There's nothing to be compacted away, and nothing to recover. Once a record has a source, what the agent believes and why shifts from something you have to argue about to something you can simply query.

2. Write First, Then Compact

Nothing is allowed to summarise or discard working context until a persistent copy exists and has been confirmed. This is a write-ahead rule — and it's the earliest of the five to take effect. If compaction runs first, whatever gets dropped is gone for good, with no error raised. It just quietly takes something with it.

This rule only holds if persistent storage is treated as the source of truth and the prompt is merely a view into it.

The Write-Ahead Log (WAL) is a classic reliability mechanism from the database world, widely used in relational databases like PostgreSQL. The core principle: any modification to data must first be persisted to disk as a log entry, and only after that write is confirmed can the actual data be changed. This way, even if the system crashes mid-operation, replaying the log can restore a consistent state — no partial writes, no silent corruption.

Applied to AI agent memory management, this means: before allowing the context window to compact (i.e., discard working content), the system must first confirm that the relevant records have been written to persistent storage. The trade-off is that compaction has to wait for the disk write to complete — on slow storage this introduces a visible delay, something the author acknowledges honestly in the limitations section.

3. Forgetting Is Bounded and Scheduled — Pinned Records Never Decay

Unbounded memory isn't a virtue. Records that are no longer relevant age out on a schedule rather than being carried forever; records you explicitly pin are exempt from decay. Without a boundary, the retrieval space grows without limit, and short queries perform worse and worse against long archives.

This is precisely the distinction between storage and a scratch pad. A scratch pad can only be cleared in bulk, whereas storage can retire records one by one with intention — and tell you what it retired.

4. Recall Returns Your Exact Words and Shows the Source

If the write was already lossy, no ranking algorithm can recover the exact phrase you actually needed. Recall fetches the raw stored record rather than a rewritten version of it, and delivers the result together with its source. This is what makes cross-session continuity real: a decision made in one session can be retrieved months later in another, with its provenance intact.

5. Privacy Boundaries Are a Setting, Not a Promise

Where a record is stored, and whether a given record is allowed to leave the local machine, should be a configuration decision — not a paragraph in a privacy policy. Local-first matters here for a concrete reason: if storage lives on your disk, compaction is a transient event you survive; if storage lives on a vendor's servers, the durability of your long-term memory depends entirely on their product roadmap.

What "Surviving Compaction" Actually Changes

Measuring against the lobotomy framing, the author outlines what genuinely shifts:

• Compaction is no longer an "event": The window can compact as often as it likes, because records were never stored only in the window to begin with.
• You can see what was remembered: Every record carries a source, so you can verify what the agent actually retained rather than guessing from a summary.
• You can correct wrong beliefs: Editing a bad record is a file operation, not a support ticket.
• Cross-session continuity is a property, not a trick: A new session can retrieve old decisions with their sources, without replaying the full conversation history.
• Memory outlives the tooling: Storage is ordinary local files that survive agent swaps, version upgrades, or vendor shutdowns.

HyperMarrow's modules — record, recall, consolidation, and file-bridge — are exposed via the MCP protocol, so agents need no custom integration per tool.

MCP (Model Context Protocol) is an open protocol introduced by Anthropic in late 2024, designed to standardize how AI models integrate with external tools and data sources. Before MCP, every tool provider had to build separate adapters for each AI platform, creating a fragmented ecosystem. MCP aims to provide a unified "socket" spec: tools expose capabilities per the protocol, models call them per the protocol, and neither side needs custom work for the other. It's analogous to what the USB standard did for hardware peripherals.

By exposing its memory modules through MCP, HyperMarrow means any agent framework that supports MCP can call its record, recall, and other functions directly — no need to rewrite integration code for each host environment. This also embodies the combination of local-first storage and protocol standardization: records live on the user's machine, but the calling interface is standardized and not tied to any specific vendor or platform.

The Limitations the Author Openly Admits

Notably, the author doesn't package this as a perfect solution. The honest shortcomings listed include:

• The write-ahead rule introduces latency: Completing a persistent write before compaction means compaction has to wait for storage. On slow disks this is a visible delay — the honest framing is that it's a trade-off, not a free win.
• Bounded forgetting can retire something you wanted: Decay is scheduled and auditable, but it's still a policy, and policies will eventually misjudge a record you needed. Pinning is the escape hatch, but it depends on you noticing in time.
• Source doesn't equal correctness: A record can carry a perfectly honest source and still be wrong. Nothing in storage will tell you that.
• Compaction isn't the only loss channel: An agent with poor summarisation, or one that writes lossy records from the start, will drop information on the way in — long before these rules ever get a chance to help.

Extend the Window, or Open the Storage?

The article closes with a question worth every agent developer's consideration: if your agent runs long enough to trigger compaction, memory is no longer a prompt-engineering problem. A longer window just moves the cliff further out.

So when your agent forgets something, would you rather raise the token budget — or open the storage and read exactly what it kept?

The author explicitly discloses that they built HyperMarrow. The community posts referenced in the article are real and public; no metrics, user counts, or testimonials were fabricated. That transparency itself lends a degree of credibility to what is, admittedly, a product-adjacent piece. For developers building long-running AI agents, the framing that "memory should be solved at the storage layer, not the prompt layer" is a perspective worth taking seriously.