Goal Leakage: How KV Cache Causes LLM Agents to Confuse Multi-Client Decisions

When an LLM agent serves multiple clients simultaneously, does it "remember" the previous client's goal in a way that influences the decision it makes for the next one? A recent study answers yes, and for the first time reveals the KV cache representations behind this phenomenon at a mechanistic level.

Goal Contamination in Multi-Client Scenarios

As LLM agents take on increasingly autonomous roles, they inevitably need to interact with multiple clients and counterparties at the same time — entities whose goals and incentives often conflict. The problem: if the agent shares reasoning state across these interactions, information belonging to one relationship can influence decisions made for another.

The research team designed a controlled two-client experimental environment with 20 market scenarios, assigning each client opposing goals. Under shared conversation memory, the model deviated from the current client's goal in 22% of eligible decisions. This means the agent isn't simply "switching" to the current client's stance — it's being continuously disrupted by residual goal representations from previous interactions.

This finding has direct implications for the design of multi-tenant LLM agent systems: shared memory improves contextual coherence, but may come at the cost of decision fidelity.

Goal Attribution Signals in the KV Cache

The study went further, examining how these goals are represented inside the KV cache. Using linear probes, the team found that client-goal attribution information becomes highly recoverable in the middle layers, and is concentrated in specific attention heads.

In other words, the model doesn't store "whose goal is this" in a distributed, hard-to-locate way — there are identifiable "hotspots." This provides a precise point of intervention: if you know where a goal representation is stored, you can potentially suppress or remove it selectively.

This result also hints at a structural feature of current Transformer architectures: the KV cache, as the carrier of reasoning state, naturally mixes information from different sources, lacking any mechanism for isolation by relationship.

KV Cache (Key-Value Cache) is the mechanism Transformer-based models use to accelerate inference during autoregressive generation: for each new token generated, the model only needs to compute the Query for the current token, while reusing the cached Keys and Values of all previous tokens to avoid redundant computation. It is essentially the model's "working memory" for the current context, containing key-value representations of all processed tokens. In multi-turn conversations or continuously running agent scenarios, the KV cache accumulates with each interaction, and information from different clients and tasks gets indiscriminately stacked in the same cache space unless explicitly cleared. This is precisely the structural root of "goal leakage" in the study — the cache itself doesn't distinguish between information sources, so goal representations belonging to a previous relationship persist as key-value vectors and get read again in subsequent attention computations, continuously disrupting current decisions.

Linear Probe is an interpretability analysis technique: a simple linear classifier is trained on the activations (or KV cache representations) of a given model layer, and its classification accuracy is used to measure whether that layer linearly encodes a certain attribute (e.g., "which client does this goal belong to"). If a linear probe can recover attribution information with high accuracy, it indicates that the attribute is linearly separable in the representation space — that is, the model stores it in a relatively explicit and concentrated way, rather than diffusing it across many dimensions. The advantage of linear probes lies in their simplicity, reproducibility, and the fact that high recoverability itself indicates the information is "readable," providing clear coordinates for subsequent causal interventions (such as targeted ablation).

Ablation Experiments: Causal Verification

After identifying the location of goal representations, the research team intervened in the cache to test whether these representations actually influence model decisions. They selectively ablated the competitor's cached goal representations, and the results showed the model shifted markedly toward the current client.

Crucially, this effect was selective: ablation on the attention heads identified by the probes had a greater effect than randomly selected heads; ablating goal tokens also had a greater effect than ablating matched non-goal tokens from the same client. This shows that goal representations aren't just correlated with decisions — they are causally responsible: removing them genuinely changes the direction of the model's behavior.

Ablation in mechanistic interpretability research refers to actively removing or masking specific activations, attention heads, or cache entries in a model and observing behavioral changes, in order to determine whether that component has a causal relationship with the target behavior. Unlike correlation-only observation, ablation establishes a causal chain through "disrupt-observe." In this study, after ablating the competitor's goal representations, the model shifted markedly toward the current client; ablation on probe-identified attention heads had a stronger effect; and ablating goal tokens had a greater effect than non-goal tokens. Together, these three points constitute causal evidence: what was removed is indeed the carrier storing that goal information, not irrelevant noise. Ablation has thus become a standard means of validating interpretability findings and providing a basis for engineering interventions.

Implications for Persistent LLM Agents

These findings point to a core problem: persistent LLM agents lack relationship-specific state isolation. When an agent maintains continuous conversations across multiple clients, goal representations in the KV cache "leak" across relationship boundaries and produce measurable effects on subsequent decisions.

From an engineering perspective, possible solutions include: maintaining independent cache partitions for each client relationship, actively clearing or masking irrelevant goal representations when switching clients, or designing attention mechanisms that can dynamically tag goal attribution. The probe localization method provided by this study offers an actionable technical path for the latter approach.

From a broader perspective, this work reminds us that an LLM agent's "memory" is not neutral information storage, but a decision component stamped with relational imprints. When designing multi-client agent systems, state isolation should not be treated as an optional optimization, but as a foundational requirement for ensuring behavioral fidelity.

One-line Summary

Research finds that LLM agents with shared memory deviate from the current client due to residual goals in the KV cache, and localizes the ablatable representations.

Paragraph Summary

A study reveals that when an LLM agent serves multiple clients with conflicting goals simultaneously, shared conversation memory leads to goal leakage: in a controlled two-client experiment, OLMo-3-7B-Instruct deviated from the current client's goal in 22% of eligible decisions. Using linear probes, the research team found that client-goal attribution information is highly recoverable in the middle layers and concentrated in specific attention heads, indicating that the model stores this information in localizable "hotspots." Further ablation experiments showed that removing the competitor's goal representations significantly pulls decisions back toward the current client, with a stronger effect on probe-identified heads, proving causal impact rather than mere correlation. The work points out that persistent agents lack relationship-specific state isolation, and proposes engineering directions such as cache partitioning, active clearing, or dynamic attribution tagging.