Google's SKILL.state Paper Explained: Replacing History with State to Save Agents 94% of Tokens

The Token Dilemma for Long-Session Agents

As AI Agents take on increasingly complex, multi-step tasks in real-world scenarios, a hidden yet critical problem has been surfacing: context bloat. During reasoning, traditional Agents carry the complete conversation history as input at every step. This means that as the number of session steps increases, the input token count grows linearly or even exponentially — every reasoning step requires re-"reading" all previous history.

To understand this, we need some background on token economics: tokens are the fundamental unit by which large language models process text, and the cost of each API call is directly tied to the number of input and output tokens. Taking GPT-4o as an example, the price per million input tokens is roughly $2.50–$5.00. For Agent systems that need to run continuously for hours or even days, cumulative token consumption can reach tens or hundreds of millions. The context window is the maximum number of tokens a model can process in a single inference call. Current mainstream models offer windows ranging from 128K to 200K, while the Gemini series has expanded beyond 1 million. But a larger window doesn't fundamentally solve the cost problem — the bigger the window, the higher the computation and expense per inference call.

In a standard multi-turn conversational Agent architecture, the input for each reasoning step consists of the system prompt, the complete message history, and the latest user/environment input. If each interaction step generates roughly 1,000 tokens, then by step N, the history portion alone carries approximately N × 1,000 tokens. In a 100-step task, this means over 90% of the input in the final steps is redundant history. To make matters worse, the self-attention mechanism in the Transformer architecture has a computational complexity proportional to the square of the sequence length (O(n²)), which means long sequences not only consume more tokens but also significantly increase inference latency.

This leads to exorbitant inference costs, slower response times, and even information loss when the context window is exceeded. For long-session Agents that need to perform dozens or hundreds of steps, this is an almost unavoidable bottleneck.

A recent paper from Google proposes a method called SKILL.state that can reduce an Agent's token usage by up to 94% in long-session scenarios while maintaining or even slightly improving task accuracy.

The Core Idea Behind SKILL.state: Replacing History with State

From "Keeping a Running Log" to "Keeping Key Notes"

The core idea behind SKILL.state is remarkably intuitive: instead of retaining the complete conversation history, maintain a structured current state.

The traditional approach is like re-reading an entire meeting transcript from the beginning before every decision. SKILL.state, on the other hand, operates more like an efficient project manager — it maintains only a continuously updated "state summary" along with the latest observation.

The "state" here is essentially a key-value structure or structured document maintained autonomously by the Agent, similar to global variables in a program or a database snapshot. For example, in a data analysis task, the state might contain fields like "loaded dataset name," "completed cleaning steps," "current analysis phase," and "intermediate calculation results." This design draws on the concepts of Finite State Machines and Blackboard Systems — the former being a classic paradigm in computational theory that describes system behavior with a small number of state variables, and the latter being an AI architecture where multiple knowledge sources collaborate through a shared workspace. By compressing verbose procedural records into a declarative current state, the model can obtain all the information needed for the next step within a constant input size.

Here's how it works in practice:

1. During reasoning, the Agent actively determines which pieces of information will be useful for future steps;
2. It writes this key information into a structured state;
3. It then discards the original conversation history;
4. For the next reasoning step, the input consists only of the updated state plus the latest observation.

This way, the input size remains essentially constant throughout the entire session, rather than growing indefinitely with each step. This fundamentally changes the Agent's context management logic: from passively accumulating history to actively distilling state.

Real-World Results: Accuracy Holds Steady — or Even Improves

The paper validated this method using Gemini-3-Flash on a 100-step benchmark, and the results were quite impressive. Gemini-3-Flash is a model variant from Google DeepMind's Gemini series, optimized for efficient inference scenarios. The Gemini series is known for its ultra-large context windows (supporting up to 1 million tokens) and multimodal capabilities, while the Flash version significantly optimizes inference speed and cost while maintaining strong reasoning ability. Choosing this model as the experimental platform has dual significance: on one hand, its large window ensures the traditional full-history approach is technically feasible (no truncation from window overflow); on the other, the Flash version's high throughput makes the token consumption comparison more practically relevant.

Method	Accuracy	Token Consumption
SKILL.state	0.94	65k
LangGraph-style stateful baseline	0.91	1.1M

In comparison, SKILL.state completed the task using only 65K tokens — a task that the traditional method required 1.1M tokens to finish — representing approximately a 94% reduction in token consumption. What's even more interesting is that its accuracy (0.94) not only didn't drop due to "discarding history," but was actually slightly higher than the baseline method that carried the full history (0.91).

This is quite revealing: more history isn't always better. Redundant context can introduce noise and interfere with the model's judgment on the current task. A distilled, structured state actually helps the model focus on what truly matters. This phenomenon echoes the "Lost in the Middle" problem observed in LLM research — when input sequences are too long, models tend to pay insufficient attention to information in the middle positions, causing critical information to be buried under massive amounts of redundant content.

Limitations and Prerequisites of SKILL.state

The Key Lies in "Anticipating Future Needs"

The paper's authors candidly acknowledge the method's core limitation: SKILL.state's effectiveness is highly dependent on the Agent's ability to accurately anticipate what information future steps will need.

Because the state is actively written by the Agent itself, if at some step it fails to recognize that a piece of information will be needed later, that information won't be written to the state. Once the history is discarded, that information is permanently lost. When it's actually needed later, the Agent can only re-retrieve or re-acquire it, potentially introducing additional overhead or even causing task failure.

In other words, this mechanism delegates the responsibility of "memory management" to the model itself. It requires the Agent to possess a degree of forward-looking planning ability — not only solving the current problem but also foreseeing what will be needed in the future. This "anticipating future needs" capability corresponds in cognitive science to the core function of working memory — selective attention and information filtering. Humans don't remember every detail when handling complex tasks; instead, they rely on metacognitive abilities to judge which information is worth retaining. In the AI field, this capability is called "learning to forget," closely related to research directions like Attention Mechanisms and Memory-Augmented Networks. Recent research shows that large language models can indeed exhibit a certain degree of information prioritization after appropriate Prompt Engineering or fine-tuning, but the reliability of this capability remains an open question.

For scenarios with clear task structures and well-defined goals, this is relatively manageable. But for highly dynamic, unpredictable open-ended tasks, the risk of misjudgment increases.

Implications for Agent Architecture Design

State Management Is Becoming a Core Competency

The value of SKILL.state goes beyond just saving tokens — it represents a paradigm shift in Agent design. Current mainstream Agent frameworks (such as LangGraph) have introduced the concept of state, but during reasoning they still often carry substantial historical context.

Take LangGraph as an example: it's an Agent orchestration framework from the LangChain team that models an Agent's execution flow as a directed graph, where each node represents an operation step and edges represent state transitions. LangGraph does introduce explicit state objects, allowing developers to define structured data passed between nodes. However, in practice, LangGraph's state is typically "incremental" — new information is continuously appended to the state, and LLM inference often still injects a large volume of historical messages as context. Similar frameworks include AutoGen (Microsoft), CrewAI, and OpenAI's Swarm, each with varying approaches to state management, but all commonly facing the same context bloat problem. SKILL.state's "aggressive discard" strategy stands in stark contrast to these frameworks, implementing the "state-first" philosophy far more thoroughly.

This offers several insights for the broader industry:

• Dramatically improved cost controllability: For production-grade Agents that need to run for extended periods, token costs are an unavoidable operational expense. A 94% reduction translates to substantial economic value.
• Breaking through context window limitations: A constant input size theoretically allows Agents to run sessions of arbitrary length without being constrained by the model's context window.
• Reallocating model capacity: The context budget saved can be used for deeper reasoning or supporting more parallel tasks.

Of course, there's still a gap between paper and production deployment. Pushing SKILL.state from academic validation to production environments requires solving several key engineering challenges. First is state schema design: different types of tasks require different state structures, and designing a template that is flexible enough without being overly complex is a practical challenge. Second is fault tolerance: when an Agent misjudges information as unimportant and fails to write it to the state, the system needs fallback strategies — such as triggering re-retrieval, calling external memory storage, or requesting user confirmation. Third is observability: when debugging and monitoring Agent behavior, discarding history means losing the complete execution trace, which poses new design requirements for logging, error tracking, and compliance auditing. Finally, there's the issue of integration compatibility with existing frameworks, since most Agent toolchains assume complete history is available.

Conclusion

SKILL.state offers a simple yet powerful insight: for long-session Agents, managing state well matters far more than accumulating history. Replacing verbose conversation histories with structured current state dramatically reduces costs while maintaining accuracy, pointing toward a pragmatic direction for building more efficient and sustainable AI Agent systems.

As AI Agents increasingly take on complex real-world tasks, context optimization techniques like this will likely become standard practice in Agent engineering. And how to teach models to "forget intelligently" may well be the defining challenge for the next generation of Agents.