Agent Memory Without Contamination: A Guide to Multi-Tier Memory Architecture Design

Give an Agent memory and it suddenly feels "alive." But if that memory has no schema constraints, it turns into the kind of colleague who treats every piece of gossip as gospel truth. Contamination is the default state: unverified tool outputs, stale preferences, and half-finished reasoning all look identical once they're embedded in a vector store.

A developer named Priya once had an Agent that confidently cited a six-month-old hallucination as fact—its memory store was packed with verified facts, outdated preferences, and unfinished analyses from last quarter, and the retrieval system treated them all the same: no tags, no source labels, no timestamp priorities. This isn't a bug—it's the inevitable result of the default architecture.

Memory Isn't a Single Bucket—It's Tiered Storage

The core insight behind production-grade memory systems is this: different types of memory must go into different tiers. According to the architecture analysis from ayraix.com, a minimally viable memory architecture contains at least three tiers: verifiable facts, human-approved decisions, and session-scoped scratch drafts. Each has different TTLs, source tags, and write permissions.

Mix all three together, and your Agent will confidently cite March's hallucinations. The internal practice pattern the author describes is: set a 7-day default TTL on unverified Agent-generated memories, after which they auto-expire—and shorten it further for high-risk scenarios. Behind this number lies a sobering industry observation: roughly 60% of Agent hallucination incidents can be traced back to contaminated memory stores, not to the model itself making mistakes.

Write Gates Matter More Than a Bigger Context Window

Reading everything into the prompt isn't memory—it's "expensive amnesia plus a bunch of extra steps." What persistent memory really needs are write gates: what content is allowed to persist, who approves it, and what metadata gets stored (source, timestamp, confidence, tool ID). The correct division of labor is—the Agent proposes memories, the policy layer commits them.

Write gates act like security guards at the entrance to each memory tier. Unverified data gets redirected to a short-TTL staging area before it can contaminate long-term storage.

TTL (Time to Live) is a core control parameter in memory systems: each record carries an expiration time at write time, and once it expires, it's automatically invalidated or archived without human intervention. In the Agent memory context, TTL is essentially a "trust time boundary"—it assumes that any information not re-verified will see its credibility decay over time. In practice, TTL tiering typically looks like this: session drafts at the hour level (e.g., 24 hours), unverified Agent reasoning at the day level (e.g., the 7 days mentioned in the article), and verified facts set to permanent or quarterly review. The hard part of TTL design isn't the setting itself—it's the post-expiration handling strategy: hard delete, soft delete (tombstoning), or demotion to cold storage? Each choice directly affects compliance auditing and dispute traceability.

Write gates draw from the database world's "validate-before-write" and the security world's "policy-as-code" philosophy. The core idea is to separate two phases: "propose" and "commit." The Agent can only produce candidate memories (proposals), while whether they actually get persisted is decided by a policy layer independent of the model. This means the gate needs at least three types of input: the source type of the content (user input, tool return, model self-generation), confidence signals about the content (is there external verification? does it conflict with existing facts?), and the caller's permission level. Gate policies can be written as rule engines or small classifiers, but they must be auditable and rollback-capable. A common mistake is implementing the gate as a prompt instruction like "please only remember important information"—that's not a gate, that's a prayer, and the model can still bypass it on the next turn.

Four Practical Engineering Patterns

Building on the write-gate tiered design, ayraix.com offers four patterns you can put into practice immediately:

• Storage isolation: user preferences ≠ organizational facts ≠ session notes—separate them physically
• Annotate sources on read: when a memory influences output, show where it came from
• Compression tasks: summarize old threads instead of endlessly replaying raw chains
• Negative memory: "we tried X and it failed" is just as valuable as success stories—tag it explicitly

The Read Path Needs Hygiene Too

Most teams agonize over what gets written to memory, but few ask: what gets read back, and in what order. Retrieval without ranking is letting a six-month-old guess override yesterday's verified fact—same embedding score, wrong recency, disastrous answer.

Memory retrieval should be policy-driven, not purely similarity-driven. The ranking dimensions the author proposes include: applying exponential decay recency weights to unverified tiers; presenting conflicting memories side by side rather than blending them; truly deleting or tombstoning when a user says "forget this"; and cross-session isolation so customer service Agents don't inherit each other's drafts.

"Memory retrieval should be policy-driven, not just similarity-driven. Boost the weight of verified sources, decay unconfirmed Agent reasoning, and hard-exclude anything marked as deprecated. Your Agent shouldn't cite a pricing figure like 'I think it's $49' from some thread—unless that thread is tagged as hearsay."

Memory you can't audit is memory you can't trust. Design the delete path with the same care as the retrieval path.

Pre-Launch Contamination Testing: The Poisoning Suite

Before deployment, build a poisoning test suite: feed the Agent false facts in session one, then ask related questions in session two. If the false information persists without source tags, your write gates have failed. Rerun this suite after every memory schema change—it's far cheaper than a customer-facing hallucination incident.

There's more you can do: instrument memory reads in your logs. When an answer goes wrong, you need to see which memory IDs influenced retrieval—not just "the model said so." Pair this with periodic compression tasks: promote recurring verified facts to long-term memory and archive noisy threads to cold storage. Memory is a database—treat it like a DBA would.

Read Ranking Strategy Example

Factor	Verified Facts	Agent Reasoning
Recency boost	Medium	High decay
Similarity weight	High	Capped
Human override	Always wins	Always wins

The "recency boost" and "similarity weight" in this table need to be tuned jointly, not set independently. A practical implementation is to express the final retrieval score as a weighted sum: score = w1 · similarity + w2 · recency + w3 · source_trust + w4 · human_override. Here, recency uses linear or logarithmic decay for verified facts and exponential decay for Agent reasoning, ensuring that distant unverified content sinks quickly. source_trust maps directly from the memory tier (verified facts highest, human decisions next, drafts lowest). When two memories have close scores and conflicting conclusions, the system should return both results side by side with a conflict flag rather than letting the ranking algorithm "decide for the user"—because blending two contradictory memories often produces answers that seem plausible but are actually wrong.

The Compliance Angle: GDPR and the Right to Be Forgotten

Memory systems must support tombstoning and propagated deletion—when a user deletes their data, any embeddings and summaries derived from it must be cleared as well. "We'll retrain later" is not compliance. Design deletion jobs that run on schedule and record their completion status.

Legal teams will ask about memory retention policy before security teams ask about injection attacks. Have answers for both. Short TTLs on unverified tiers aren't just a quality issue—they're regulatory hygiene.

Compression Mustn't Become Amnesia

Thread compression saves tokens, but it carries the risk of summary drift—each round of summarization can lose nuance. The author's recommendation: store raw logs in cold storage for dispute resolution, and give the Agent a compressed view. When legal asks "what did the bot know on March 3rd," you need the raw archive, not a summary of a summary.

• Versioned summaries: tag compression generations and roll back when quality degrades
• Pin critical facts: promoted content survives compression; chitchat doesn't
• Promotion requires human review: facts entering long-term tiers need a human to sign off

Treat Memory Like a Database, Not a Diary

Start with a three-tier architecture—verified facts, approved decisions, temporary drafts—and specify an owner, TTL, and delete path for each tier before allowing any Agent to write to it. The poisoning test suite isn't optional homework; it should run after every schema change, just like production table migration tests.

The Agents that earn trust aren't the ones that remember the most—they're the ones that can tell you exactly which memory ID caused a wrong answer, and delete it in one step. Source tracking and tombstoning are the whole ballgame; everything else is just embeddings with extra confidence scores.

Action items: Split your current memory store into three buckets—facts you'd defend in court, helpful-but-possibly-wrong notes, and drafts that should evaporate by Friday. Add a provenance field and ttl_hours to every record. Write a one-hour poisoning test. That's the entire architecture; everything else is optimization built on top of hygiene—and hygiene is the only thing users actually feel when they get a wrong answer.