Agent Context Engineering: Caching, Compaction, and Subagent in Practice

When AI Agents move from demos to production, the biggest bottleneck developers encounter is often not model capability, but context management. Context windows are limited, while an Agent's conversation history, tool call results, and retrieved documents keep expanding. How can an Agent run efficiently, accurately, and cost-effectively within a limited window? The answer lies in Context Engineering.

Both Anthropic and OpenAI provide key control mechanisms at the API level. This article breaks down five core strategies—caching, compaction, tool-result clearing, retrieval, and subagents—and provides actionable API parameters.

Why Context Engineering Is the Core Challenge of Agents

The essential difference between an Agent and an ordinary chatbot lies in multi-turn tool calls. Every tool call produces a result, and these results are appended to the conversation history. A complex task may involve dozens of tool calls, each returning JSON or text that can be thousands of tokens long. Soon, the context window is filled up.

The severity of the problem manifests at three levels: cost—billing is by token, so redundant context directly drives up expenses; latency—longer context means slower inference; quality—irrelevant information dilutes attention, making the model more error-prone. The goal of context engineering is to find a balance among these three.

Strategy 1: Prompt Caching—Stop Paying for Repeated Prefixes

Prompt Caching is the most direct means of cost optimization. The core idea is simple: if multiple requests share the same prefix (such as system prompts, tool definitions, long documents), cache the computation results of that part to avoid reprocessing.

Anthropic's implementation: Mark cache breakpoints via the cache_control field. You can set {"type": "ephemeral"} on system prompts, tool definitions, or message blocks. Cache-hit tokens are billed at a lower price, while cache writes incur an additional fee. This suits scenarios where system prompts are stable but conversations continue.

OpenAI's implementation: Automatically caches prompt prefixes exceeding 1024 tokens, with no explicit marking required. Input tokens that hit the cache receive a discount. Developers only need to keep the prompt prefix stable to benefit.

The difference lies in control granularity: Anthropic allows developers to precisely specify cache locations, while OpenAI is more automated. A practical recommendation is to place stable system prompts and tool schemas at the very front and dynamic content at the back to maximize cache hit rate.

How cache breakpoints work: In Anthropic's API, a cache breakpoint marked with cache_control treats all preceding content (including system prompts, tool definitions, and previous messages) as a cacheable prefix. The cache is valid for 5 minutes by default, during which requests with the same prefix can directly reuse the computed results. Writing to the cache incurs an additional fee of about 25%, while cache-hit tokens are billed at only 10% of the base price. Therefore, for multi-turn conversations or frequently invoked Agents, caching can significantly reduce costs. Note, however, that the cache prefix must be exactly identical; any slight change (such as a timestamp or random ID) will invalidate the cache.

OpenAI's automatic caching: OpenAI automatically enables caching for prompts exceeding 1024 tokens, with no developer intervention required. Cache-hit input tokens are billed at a 50% discount. Its cache is based on a hash of the prompt prefix, which likewise requires a stable prefix. Developers can maximize hit rate by placing static content first and dynamic content later. The core difference between the two lies in control granularity: Anthropic allows precise control over cache locations, suitable for complex scenarios; OpenAI is more automated but less flexible.

Strategy 2: Compaction—Folding Historical Conversation into a Summary

When conversation history grows too long, compaction becomes necessary. The approach is to have the model summarize early conversation into a concise summary, replace the original history with the summary, and retain only the most recent few rounds of complete conversation.

This is essentially a form of lossy compression: details are lost, semantics are retained. The key lies in summary quality—a good summary preserves task goals, completed steps, key findings, and unresolved constraints; a poor summary loses critical information, causing the Agent to duplicate work or make wrong decisions.

In implementation, a dedicated summarization call can be triggered when the history length exceeds a threshold, compressing the first N rounds of conversation into a single system message. Anthropic's context window management documentation recommends retaining the most recent tool call results and the original user instructions, compressing only the intermediate process.

The trade-off of lossy compression: Compaction is essentially lossy compression that may lose details while reducing tokens. To balance information retention and compression ratio, a hierarchical summarization strategy can be adopted: first generate a coarse-grained summary of early conversation, then a fine-grained summary of recent conversation, forming a summary of summaries. Another approach is to retain the original text of key entities (such as user goals, constraints, and confirmed facts) and compress only the reasoning process and tool call details. Anthropic recommends triggering compaction when context usage reaches 70%-80%, avoiding rushed processing near the window limit. The compacted summary should be inserted as a system message and clearly labeled "the following is a historical summary" to distinguish it from the original conversation.

Strategy 3: Tool-result Clearing—Discard Tool Results Once Used

Tool call return results are often the main source of context bloat. A web scrape may return tens of thousands of tokens of HTML, and a database query may return a large number of rows. But once the Agent has extracted the key information, the original content is no longer needed.

The Tool-result Clearing strategy is: after the Agent processes the tool result and extracts the required information, remove the original result from the context or replace it with a short marker. This is more aggressive than Compaction, but also more efficient.

The risk is that if subsequent steps need to refer back to the original result, it cannot be recovered after clearing. Therefore, it is necessary to determine which tool results are one-off (such as search result summaries) and which may need repeated reference (such as configuration file contents).

Granularity control of clearing strategy: Tool-result Clearing is not simply deleting all tool results, but requires granularity control based on tool type and subsequent needs. For example, for search results, retain titles and summaries and delete full web page content; for database queries, retain aggregate statistics and key rows and delete the raw dataset. In implementation, an ephemeral marker can be added to tool definitions to indicate that the tool result can be cleared after extraction. At the same time, metadata of tool calls (such as call time, parameters, and result summary) should be retained for traceability. For results that may need to be referred back to, store them in external storage and keep a reference ID in the context, retrieving them via a retrieval tool when needed.

Strategy 4: Retrieval—Fetch on Demand Rather Than Loading Everything

Rather than stuffing all potentially relevant documents into the context, let the Agent actively retrieve when needed. The Retrieval strategy decouples the external knowledge base from the context, and the Agent obtains snippets on demand through tool calls.

This requires a well-designed retrieval tool: query interface, return format, and snippet granularity. Returned results should include sufficient metadata (source, relevance score) to help the Agent decide whether further retrieval is needed. Anthropic's Agent building guide emphasizes that retrieval tools should return concise snippets rather than entire documents, and support pagination or filtering.

Retrieval and Tool-result Clearing are naturally complementary: retrieval results can be cleared once extracted, keeping the context lean at all times.

Strategy 5: Subagents—Isolate Complex Subtasks with Independent Contexts

When a subtask requires a large number of exploratory tool calls, rather than letting the main Agent's context be polluted by these processes, launch a subagent to complete the task in an independent context and return only the final result to the main Agent.

The value of Subagents lies in context isolation. A subagent can freely call tools, experiment, and retrieve, and its lengthy intermediate process will not affect the main Agent's decision space. The main Agent receives only a concise conclusion.

The cost is coordination overhead: clear input-output contracts need to be defined, subagent failures need to be handled, and state synchronization among multiple Agents needs to be designed. Anthropic's multi-Agent research system demonstrates the effectiveness of this pattern in complex research tasks, but also notes that token consumption increases significantly.

Communication contract for subagents: A clear input-output contract is needed between the main Agent and subagents. Input should include the task description, available tool list, constraints, and expected output format; output should be a structured result (such as JSON) containing conclusions, confidence, and cited sources. The subagent's context is completely independent and does not inherit the main Agent's history, so sufficient background information must be provided in the input. The main Agent should set timeouts and retry mechanisms to handle subagent failures. In addition, the subagent's token consumption counts toward total cost, and its usage efficiency should be monitored. Anthropic's research shows that the subagent pattern is notably effective in tasks requiring multi-step exploration, but it should be avoided for simple tasks to prevent unnecessary overhead.

Combined Use: No Silver Bullet, Only Trade-offs

The five strategies are not mutually exclusive; production-grade Agents typically use them in combination. A typical architecture is: Prompt Caching for system prompts and tool definitions; periodic Compaction for long conversations; Clearing after tool result extraction; Retrieval for external knowledge on demand; and Subagents to isolate complex subtasks.

Each strategy has a cost: caching has write costs, compaction has information loss, clearing has backtracking risk, retrieval has latency, and subagents have coordination overhead. The essence of context engineering is engineering trade-offs among cost, latency, and quality, rather than optimizing for a single metric.

Only by understanding these API-level control mechanisms can Agents run stably in real scenarios, rather than crashing when the context overflows.

One-line Summary

Five context engineering strategies help Agents balance cost, latency, and quality within a limited window.

Paragraph Summary

This article explores the context management challenges faced by production-grade AI Agents and systematically breaks down five core strategies: Prompt Caching reduces cost and latency by caching repeated prefixes; Compaction lossily compresses historical conversation into summaries; Tool-result Clearing removes original tool results after information extraction; Retrieval fetches external knowledge on demand rather than loading everything; and Subagents isolate complex subtasks with independent contexts. The article compares the differences between Anthropic and OpenAI in cache implementation and points out that each strategy has costs and must be used in combination to balance cost, latency, and quality.