Context Poisoning: The Real Culprit Behind AI Agent Failures Is Data, Not the Model

When an AI agent is asked to look up all of someone's sales calls since February 1, it may return calls that person never participated in—even though the correct call records are right there in its context window. The problem lies in the data reaching the model, not the model itself.

Airbyte CEO and co-founder Michel Tricot ran exactly this query against the Gong API in a live demo. Because the API can't filter calls by user, the agent had to page through roughly a month and a half of all call records. Tricot says this path typically takes about three minutes, and because the bloated context prevents the agent from narrowing down to the right person, it often returns more than 45 calls and consumes nearly 30,000 more tokens than the path through context storage. These are his rough numbers from a single demo, not a benchmark, but the shape of the problem is very common: the tool offloads filtering work onto the model that should have been done by the application.

This is "Context Poisoning"—wrong, stale, or poorly structured data enters the context window, the agent reasons over it, and it errs without any malicious behavior involved.

As everyone says, the problem today isn't the model—it's the data you feed the model.

— Michel Tricot, Airbyte CEO and co-founder, Chain of Thought Episode 52

Log at the Insertion Point

Instrument at the point where tool results become model-visible text. Logging HTTP response size alone isn't enough, because any filtering or transformation in between changes what the model actually sees.

For each tool call, log the number of tokens added by the serialized result and the record IDs it carries. After the run, record how many of those records the answer actually used. Keep the exact payloads (under normal data access controls) so you can read what the model read.

import json

def append_tool_result(call_id, tool, records, messages, audit):
    content = json.dumps({"records": records}, separators=(",", ":"))
    messages.append({"role": "tool", "tool_call_id": call_id,
                     "content": content})
    audit[call_id] = {
        "tool": tool,
        "tokens_added": count_tokens(content),
        "returned_ids": {r["id"] for r in records},
        "ids_used": None,  # unknown until someone checks the answer
    }

def record_usage(call_id, verified_ids, audit):
    entry = audit[call_id]
    entry["ids_used"] = len(entry["returned_ids"] & set(verified_ids))

Call this wrapper after all data shaping, so what you log matches exactly what the model receives. verified_ids should be filled in from checked citations or review—citations show what the model claims it used, not necessarily what it actually relied on, so the field should remain unknown until someone verifies it.

Context Window refers to the upper limit of input tokens a model can hold in a single inference pass; all system prompts, user messages, tool results, and conversation history share this budget. Exceeding it means either truncation or an error, and truncation is often exactly why an agent loses key records. A Token is the smallest unit of text a model processes—roughly three-quarters of a word in English, while a Chinese character usually takes 1 to 2 tokens. A Tool Call is the mechanism by which an agent requests an external function to execute via a structured message and returns the result; the returned content enters the conversation history as a message with role: "tool", permanently occupying context from then on. A Payload is the raw data body a tool actually returns. Only by understanding the relationships among these four can you see why every tool result keeps consuming the token budget of every subsequent turn—and that's the fundamental reason insertion-point logging must capture token deltas rather than just HTTP size.

Precision and Recall describe retrieval quality in this context: recall is the ability to find all relevant records, while precision is the proportion of relevant records among the returned results. Context poisoning is usually a precision problem—the relevant records may all be there, but they're drowned out by a mass of irrelevant ones, making it hard for the model to weight them correctly.

Focus on Precision, Not Just Size

A large result isn't necessarily a bad result—comparing 40 calls genuinely requires 40 calls.

The real warning sign is when a large result is full of records outside the scope of the question. Near-duplicate records crowd the context window, and the model may be unable to give the correct records enough weight even when it has them. This kind of trace reads like a reasoning failure, leading teams to evaluate the model when the problem is actually at the input.

For failed runs, compare the IDs passed against the records the task required, and check how any erroneous claims relate to adjacent records in the payload. Then re-run the task with shaped results, the same model, and the same prompt, checking correctness and coverage while watching whether the context window shrinks.

Filter Before the Model Sees It

Tricot describes doing this work on the retrieval side:

The first thing is that a lot of the work happens in the backend, making sure we only return what we believe is the most relevant information.

— Michel Tricot, Airbyte CEO and co-founder, Chain of Thought Episode 52

Push task constraints down to the source where it can support them: fetch by ID, filter by user, apply date ranges. Return only the fields the question needs, plus identifiers for attribution. When the source can't express the query, filter in application code or query a store synced for retrieval. Don't send every page to the model just because the upstream API happens to paginate.

Projection is task-specific. A call-list tool can return compact metadata. A question tool that answers what someone said during a call needs the relevant transcript—trimming it saves tokens but leaves the answer unsupported.

Projection borrows from database terminology, meaning selecting only some fields from a record to return rather than the entire row. It reduces both token consumption and distracting information. Pushdown means pushing operations like filtering, sorting, and aggregation as close to the data source as possible (API parameters, SQL WHERE clauses), avoiding pulling raw data up to the application layer for processing. When the upstream API supports owner_id or date-range parameters, pushdown can significantly shrink the returned set; when the API doesn't, the next best thing is to do it in application code or a retrieval store—but never hand unfiltered full data to the model. A retrieval-synced store means periodically syncing source system data into a local vector store or document store to support query shapes the source can't express, at the cost of introducing freshness problems—which is why the next section discusses live reads.

Decide Which Reads Must Be Live

Synced stores introduce freshness problems. Tricot's own approach is a hybrid pattern: search and discover in the store, then go to the API to fetch specific objects when the task needs the latest version.

APIs are very good at retrieving a single object—as long as you know what that object is.

— Michel Tricot, Airbyte CEO and co-founder, Chain of Thought Episode 52

Make this a tool policy rather than letting the agent improvise.

FIELDS = ("id", "owner_id", "started_at", "title")

def get_call(call_id, require_live):
    source = "live_api" if require_live else "synced_store"
    record = fetch_record(source, call_id, fields=FIELDS)
    return {f: record[f] for f in FIELDS}

The final projection controls what enters the context window, even if the upstream response carries more information. Exposing the sync timestamp makes staleness visible. A task that needs a complete, up-to-date list needs a discovery path that's equally up to date.

Freshness refers to the time gap between the data in a store and the latest state of the source, usually measured by a sync timestamp. Tolerance for freshness varies enormously across tasks: discovery tasks (listing all of someone's calls) can accept a few hours of delay, while reading a single record's content and needing the latest version may require second-level real-time data. The Hybrid Pattern is designed precisely to balance these two types of needs—using a synced store for broad search and filtering, then going back to the API for a few key objects to get their real-time versions. The key is to bake this policy into the tool interface (for example, expressing it explicitly via a require_live parameter) rather than letting the model guess which path to use on each call, because the model can't reliably judge how stale the data is.

Key Takeaways

For your next agent run failure:

• Read the exact tool payloads that entered the context
• Log tokens added per tool call and the records the answer used
• Filter and project before insertion, not after
• Set separate freshness rules for discovery and single-record reads

A more detailed explanation (with show clips) is available at Chain of Thought.

One-line Summary

AI agent failures are mostly caused by context poisoning—the problem is the data fed to the model, not the model itself.

Paragraph Summary

The article centers on "context poisoning": in an Airbyte CEO Michel Tricot demo, because the Gong API couldn't filter by user, the agent had to traverse a month and a half of all calls, returning more than 45 irrelevant records and consuming nearly 30,000 extra tokens—confirming the root cause of failure is data, not the model. The author proposes four steps: first, instrument at the insertion point where tool results become model-visible text, logging token deltas and returned/used record IDs; second, focus on precision rather than result size in failed runs, comparing passed IDs against required records; third, filter and project before the model sees data, pushing constraints down to the source or application code; fourth, set separate freshness rules for discovery paths and single-record reads, using a hybrid pattern of store search plus live API fetch.