Inside Redis's Playbook: Context Engineering for Production-Grade AI Agents

When building production-grade AI agents, context engineering often matters more than model selection. In a recent talk, Tyler Hutcherson, head of Redis's Applied AI engineering team, and AI engineer Andrew Perukin drew on their hands-on experience with SRE agents and more to dig into how to design, manage, and optimize an agent's context window.

What Is an Agent's Context?

Andrew likens context to human "working memory": when you juggle multiple tasks, working memory supplies all the information you need for the task at hand. For an LLM, context is the data sent along with each API call—system prompts, tool descriptions, live logs, metrics, and cluster state snapshots.

The key point: context is not static information but a constantly shifting stream of dynamic data. When an SRE agent troubleshoots a Redis cluster issue, it needs to watch logs, metrics, user behavior, and the cluster's real-time state all at once. But when designing the system, developers often can't predict which information will actually be relevant—which raises the core tension: how do you sift the most valuable snippets for the current task out of a sea of data?

Why "Stuffing the Context" Fails

When faced with a context window, a common mistake is to cram in every piece of potentially relevant data. Tyler and Andrew explicitly argue against this, for two reasons.

Attention dilution and the "needle in a haystack" problem. If the context contains one critical error log but is mixed with thousands of irrelevant logs, the attention mechanism struggles to pinpoint that needle. Worse, the model may be misled by irrelevant information and go solve the wrong problem. Andrew stresses that we want the model to answer "given this error and the current cluster state, what's the most likely fix?"—not to flounder in noise.

Runaway costs. Filling the context window on every request means token consumption skyrockets. Tyler admits that without controls, the team's budget could burn out fast—even reaching the awkward point where "we spend more on tokens than on salaries." Real-world cost constraints directly force context engineering optimization.

Technical background on attention mechanisms and context windows: Modern LLMs rely on the self-attention mechanism in the Transformer architecture—when processing each token, the model computes relevance weights against every other token in the context. When the context window is packed with irrelevant information, attention weights get spread across a large number of low-relevance tokens, diluting the signal of key information—that's the technical root of "attention dilution." Moreover, although mainstream models' context windows have expanded to the hundreds-of-thousands-of-tokens level, the effective utilization length (the range of information the model can reliably attend to) is often far smaller than the nominal limit, and there is a "lost in the middle" phenomenon: information in the middle of a long context is more easily overlooked than information at the beginning and end. This explains why simply piling on data not only wastes money but also degrades reasoning quality.

Tool Abstraction and Security Boundaries

In designing the SRE agent, the team did not give the LLM direct access to the database. Instead, they built an abstraction layer: the agent can only interact with the system through predefined tools. This yields two benefits: security checks can be enforced, and detailed audit logs can be recorded.

Tyler notes that model providers are internalizing tool-use capabilities into training. If you give the model a prompt with a long list of tools, each with only a brief description, the model easily produces a flood of incorrect tool calls, and those bad calls accumulate in the context, further polluting subsequent decisions. Progressive disclosure of tools thus becomes the better option: dynamically surface relevant tools based on the task stage rather than showing everything at once.

Tool use/function calling and progressive disclosure: Tool use refers to the LLM outputting structured function-call requests (e.g., tool names and parameters in JSON) when generating a response, which an external runtime executes and returns to the model, forming a "reason-act" loop. Model providers inject tool-use data during training so the model has some calling ability, but when the tool list is too long, the model must choose among many options and easily produces hallucinated calls (calling nonexistent tools or passing wrong parameters). Progressive disclosure borrows from the principle of progressive disclosure in human-computer interaction: start with a high-level, small, precise tool set, then dynamically load relevant tools once the task reaches a specific stage. This significantly reduces the candidate space for each decision, cuts down on erroneous calls and their contamination of the context, and makes it easier to apply differentiated permissions and audit policies by stage.

Evaluation and Observability: How Do You Know It Works?

When the team tries to optimize context strategies, a fundamental question surfaces: how do you tell whether a change works? Both Andrew and Tyler stress the necessity of evals. Tyler calls evaluation "boring but essential"—like writing unit tests: nobody enjoys it, but without it you can't iterate safely.

Beyond evaluation, traceability is equally critical. What the agent saw during a run, which tools it called, and which information it based decisions on—all of this needs to be recorded and tracked. Andrew mentions that audit events and lineage tracking help teams understand "why the agent made this choice," especially in multi-turn conversations and conflicting dynamic events, where tracing ability directly affects debugging efficiency.

Engineering practices for evals and traceability: Evals means building repeatable test sets and scoring criteria for agent behavior, typically including offline evaluation (batch runs with labeled data or simulated environments) and online evaluation (A/B or shadow testing on real traffic). For context engineering, evals need to cover multiple layers of metrics: "is the retrieved context relevant," "are the tool calls correct," and "is the final answer accurate." Traceability relies on structured logging and distributed tracing: recording the complete input (system prompt, tool list, retrieved snippets), output, tool-call chain, and latency of every LLM call to form a replayable execution trajectory. Lineage tracking further records the source and flow path of each piece of context data, so that when the agent makes a bad decision, you can quickly determine whether the problem lies in the data source, the retrieval strategy, or the prompt design—enabling safe iteration.

Automatic Optimization of Agent Skills: Reality and Limits

At the end of the talk, Tyler raised a frontier topic: can agents automatically learn from past sessions and optimize their own skills? His judgment: the skills themselves are real and useful, but the "automatic optimization loop" is still in its early days.

A viable workflow is: record the agent's complete execution trajectory, generate skill-improvement suggestions from these logs, but have humans review and apply them in the end. Tyler makes clear he would not let an LLM autonomously build and use skills without review. As for when a fully automated loop will be achievable, there's no consensus yet.

Conclusion

The Redis team's practice reveals a core trend: the competitiveness of production-grade agents is shifting from model capability to context engineering capability. How precisely you retrieve context, represent it efficiently, expose it safely, evaluate it continuously, and trace it will determine whether an agent succeeds or fails in real business scenarios. For teams building AI Agents, these frontline lessons are more valuable than any theoretical framework.

One-line Summary

Redis's team shares frontline context engineering for production-grade AI agents: precise filtering, tool abstraction, evals, and traceability make or break the outcome.

Paragraph Summary

Drawing on hands-on experience with SRE agents, Redis's Applied AI engineering team systematically lays out context engineering for production-grade AI agents. The article likens context to an LLM's working memory, noting that stuffing the context window causes attention dilution and runaway costs, and stressing the need to filter relevant data precisely. On security, the team restricts LLM access via a tool abstraction layer and uses progressive tool disclosure to reduce erroneous calls. Evals and traceability are treated as the foundation of iteration, used to judge whether context strategies work and to trace decision-making. Finally, the article explores automatic optimization of agent skills, arguing that skills are useful but a fully automated loop still needs human review and remains at an early stage.