How Context Eviction Mechanisms Made an AI Agent Spin Its Wheels for an Hour

An autonomous agent spent a full hour researching a codebase, executing 126 tool calls, and wrote not a single line of code. It wasn't hallucinating, nor was it slacking off—it simply couldn't access the contents of files it had just opened seconds earlier.

This is a real infrastructure failure encountered by Legion, a multi-agent collaboration hub, while developing a Go project. The root cause wasn't insufficient model capability, but rather a deadlock formed when two independently "correct" mechanisms collided.

Background: Why a Context Gateway Is Needed

Any developer working with LLM agents knows that when an agent operates autonomously on a codebase, context balloons dramatically with every tool call. Reading files, running greps, executing tests—all history gets stuffed into the next model request. The cost of this loop grows almost quadratically: each step's results are billed again in every subsequent iteration.

Legion's agent teams collaborate in virtual "rooms," sharing cloud-based code repositories. To control costs, developers designed a deterministic middleware layer in July, sitting between the tool execution loop and the model provider, composed of three components:

Observations: Full results of each tool execution are stored in external storage, with only lightweight references retained in context.

Receipts: Stale results in requests are replaced with brief stubs containing tool name, purpose, line ranges, hash values, and the number of evicted tokens.

Read Ledger: The system tracks which file ranges the agent has already received, returning receipts instead of full text when unchanged fragments are read repeatedly.

The default eager policy retains raw text only for the three most recent tool results, capped at 6000 tokens. Offline replay on 29 anonymized trajectories showed a 53% reduction in total inbound traffic—the numbers looked beautiful.

Prompt Caching is a cost optimization mechanism offered by mainstream LLM providers (such as Anthropic, OpenAI): when the prefix portion of multiple requests is completely identical, the provider caches the computation results for that portion, and subsequently cached tokens are billed at a rate far below normal input pricing. The "cache volume" mentioned in this article refers to the number of input tokens that hit the provider's cache, while "uncached volume" refers to the incremental portion that must be recalculated each time.

Understanding this mechanism is crucial for reading the data that follows: if developers simply crudely trim context, it may appear to reduce total tokens, but as long as the cache prefix is continuously broken, the absolute volume of the uncached portion may actually increase, and the actual bill may not decrease. The value of Legion's fix lies precisely in the fact that both cache volume and uncached volume decreased simultaneously, indicating that the agent genuinely needed less information—rather than simply shifting costs from one place to another.

The Failure Scene: An Hour-Long Zero-Output Death Loop

One of Legion's backend agents was assigned the task of adding a kanban endpoint to a Go backend. This was already the third retry. After an hour, the scheduler forcibly terminated the session due to timeout.

The logs showed: 126 tool calls (99 bash, 18 observation reads, 9 read_file), zero code modifications. The agent's internal reasoning exposed its predicament:

"Let me look at the evicted repository mapping output"—it noticed the eviction almost immediately.

"This is strange—normally read_file returns content. The receipt indicates the tool result was evicted"—after realizing it received a receipt rather than code, the agent tried to bypass the restriction using cat.

"I've read it twice, and both times it was evicted from context"—the classic infinite read loop.

"I've already spent a lot of budget. Let me be decisive"—panic escalating, but it fundamentally lacked the fresh signatures needed to write code.

Root Cause Analysis: A Fatal Collision of Two Correct Mechanisms

The investigation revealed a textbook deadlock: two independent logic flows each operating normally but inconsistent with each other.

Problem One: Always-on Eviction vs. On-Demand Eviction. The context assembler ruthlessly converted any file read from more than three calls ago into a receipt, even when the window had abundant free space (say, only 20K used out of 100K tokens). During development, the team chased context window optimization, overlooking that offline replay calculated savings rather than task success rates.

Problem Two: Room-Level Deduplication Stacked on Session-Level Retention. Observation artifacts were retrieved using the key room + tool + params + file fingerprint. As a result, when the second and third attempts to solve the task hit the cache from the first attempt, they were marked as duplicate reads and returned receipts again.

A vicious cycle formed: context told the agent "there's no text here, re-read it," while the Read Ledger intercepted it with "you already got this, here's a stub." The agent retreated to cat, aggressive eviction triggered again—the file vanished once more. Meanwhile, the manifest was interpreted by the model as a direct prohibition ("don't re-read").

Context Window refers to the maximum number of tokens a model can process in a single request, including the sum of system prompts, historical conversation, tool results, and current input. Every model has a hard limit (such as 200K tokens); when exceeded, it either errors out or the external "context assembler" must actively trim, summarize, or evict old content.

The key contradiction in this article stems precisely from this limit management. The assembler treated "evict after N calls" as a universal safety rule, but this rule doesn't perceive the window's actual utilization—even if a 100K token window is only using 20K, it evicts anyway. This "capacity anxiety" seemed conservatively prudent at design time, but in practice it prematurely cleared valuable, still-fresh code information, forcing the agent to repeatedly re-read, creating a counterintuitive negative optimization. Understanding the window model helps clarify: eviction strategies must be capacity-driven, not step-driven.

The Fix: Four Key Changes

The post-mortem drove four adjustments to the gateway architecture:

Switch to on_demand eviction mode. Data is only evicted when the total request volume genuinely exceeds budget limits, and eviction stops strictly upon returning to normal range. The old behavior was preserved as an optional comparison mode.

Content retention tied to session. Cross-session caching of observation records was retained to save space, but the decision of "return text or receipt" is strictly based on whether the current session has seen that result. New attempts start from a blank state.

Honest manifest text. Stub wording was rewritten: instead of prohibiting operations, it explicitly states that text is not in context and permits re-reading files.

Synchronized with compression operations. The assembler now explicitly informs the read ledger which text blocks were deleted during overflow, resetting the "already read" status of erased fragments.

Read Ledger was designed for deduplication: when an agent reads the same fragment of the same file multiple times, the system judges it as redundant work and returns a lightweight receipt marked "already read," thereby avoiding repeatedly stuffing identical content into context and wasting tokens. This is entirely correct in the ideal scenario of a single session with unchanged files.

The problem lies in the fact that the determination of "whether the same file counts as duplicate" crossed session boundaries. The cache key includes "room + tool + params + file fingerprint," but different task retries represent different session intents. The agent at this point has no old text remaining in context, yet the ledger still tells it "you already got this." The "content retention tied to session" in the fix is essentially narrowing the deduplication scope from room-level back to session-level, letting the cache serve only genuinely shared knowledge bases without overstepping to decide whether an agent can currently see file contents.

Data Validation: A Dual Leap in Quality and Efficiency

After switching to the new architecture, comparing average metrics between September and October:

Prompt Caching: Average cache volume dropped from 6.48M to 3.48M (-46%), average uncached volume dropped from 1.71M to 1.03M (-40%). Absolute volumes nearly halved while ratios remained constant, indicating the load reduction stems from the agent genuinely needing less context to complete work, rather than surface-level cache optimization.

Output structure changes:

• Prompt: 8.01M → 5.51M (-31%)
• Reasoning: 76.1K → 58.2K (-24%)
• Completion: 62.2K → 35.8K (-42%)
• Completion/Reasoning ratio: 0.82 → 0.65 (-20%)
• Reasoning/Prompt ratio: 9.28 → 11.2 (+21%)

Two key signals are worth noting. First, verbose output collapsed: in September, final response volume nearly equaled reasoning volume—models lacking contextual facts improvised code in responses to compensate for memory gaps; in October, the agent began thinking deeply and answering concisely. Second, desperation spikes disappeared: in September samples there was an anomalous case (15.4M prompt / 152K reasoning / 111K completion) reflecting deep recursive agent collapse; in October, such extreme outliers vanished entirely.

Why Go Development Transformed Qualitatively

The ultimate proof of correctness manifested in actual software engineering. The Go language is extremely strict, idiomatic, and devoid of dynamic magic: it relies on explicit structures, rigorous error handling (if err != nil), context.Context propagation, and typed configuration.

In September, receipts erased underlying details; the agent lost the structure of production files and neighboring packages when writing configuration or services, wiped out ports and secrets when rewriting configuration from scratch, or generated code that clashed with the project.

In October, retaining real code in context pushed In-Context Learning (ICL) to its extreme. The agent switched to precise imitation mode: seeing real signatures, replicating architectural layouts (handler -> service -> repository), cleanly embedding new microservices into the existing monolith without breaking dependencies.

In-Context Learning (ICL) refers to a model's ability to adapt to tasks without fine-tuning, relying solely on examples or real code provided in the prompt at inference time. Its effectiveness depends heavily on the fidelity of context: when signatures, interfaces, and directory structures appear in their original form, the model can directly imitate them; once these details are replaced by summaries or stubs, the model can only fill in gaps using statistical priors from training data.

This is why languages like Go are particularly sensitive to context quality. Go has no dynamic magic; interfaces, error propagation, and package dependencies all rely on explicit declarations. A single erased struct field or context.Context parameter causes compilation failure. Meanwhile, the Go community's code style is highly convergent (if err != nil, handler -> service -> repository layering), so once the model sees real neighboring code, it quickly aligns with idiomatic patterns. The combination of these two characteristics makes "retaining real code" far more beneficial in Go projects than in permissive languages.

Five Lessons for Agent System Developers

Evict on demand, not on schedule. Fixed rules like "delete everything from N steps ago" disrupt workflows when the window still has plenty of free space.

Isolate session context. Knowledge bases and artifact caches can be shared across projects, but the decision of whether to show text to an agent must be strictly scoped to its individual session.

Speak honestly to the model. Instructions like "don't re-read" paralyze LLMs. Rewriting to "the text is not in context; you may call the tool again if needed" restores its freedom of action.

Token metrics are not success metrics. Offline optimization targeting input/output volume can silently kill an agent's ability to solve real problems.

Read reasoning logs. Phrases like "evicted twice" appearing in logs are early signals of systemic failure, far earlier than timeout triggers.

Trying to save a few hundred thousand input tokens with questionable stubs instead led to wasting millions of tokens on endless debugging and hallucination. The balance point found (on-demand eviction + honest receipts) elevated autonomous agents from hesitant junior engineers to reliable mid-to-senior engineers.

One-line Summary

Aggressive context eviction colliding with cross-session deduplication caused the agent to spin idle; switching to on-demand eviction with session isolation fixed both efficiency and quality.

Paragraph Summary

Legion's multi-agent system encountered a bizarre failure while developing a Go project: a backend agent executed 126 tool calls in one hour with zero code output. The root cause was the collision of two independently correct mechanisms—the context assembler unconditionally evicted file contents from more than three calls ago, while room-level deduplication hit old session caches and returned receipts, trapping the agent in a death loop of "can't read, and not allowed to re-read." Fixes included switching to on-demand eviction, binding content retention to sessions, rewriting stubs with more honest wording, and synchronizing with compression to reset read status. The results were dramatic: average cached tokens dropped 46%, completion output dropped 42%, and the model shifted from verbose compensation to concise responses. The article emphasizes that token savings cannot be equated with task success rates, and reading reasoning logs is an early means of detecting such systemic failures.