SelfCompact: Letting Language Models Decide When to Compact Their Own Context

Title

SelfCompact: Letting Language Models Decide When to Compact Their Own Context

Meta Description

SelfCompact proposes letting language models autonomously decide when and at what granularity to compact context, replacing fixed token-threshold schemes and preventing long-trajectory agents from losing partial results mid-reasoning or mid-search.

Article Content

For long-horizon agents, the "context bloat" problem has long been patched over with fixed-threshold compaction—but thresholds ignore task structure, and can drop intermediate results right in the middle of your reasoning. SelfCompact takes a different approach: let the model decide when and how to compact.

Where the Problem Lies: The "Anchoring Effect" of Long Trajectories

An agent's trajectory is a stack of chains of thought and tool calls. As the number of steps grows, the history fills with large amounts of stale content, and this content keeps "anchoring" subsequent generation—the model tends to continue along the erroneous direction it has already written down, rather than re-evaluating. At the same time, the trajectory will eventually exceed the physical limit of the context window.

Existing scaffolds handle this with fixed-interval compaction, or by triggering summarization when some token threshold is hit. The problem with this approach is that it pays no attention whatsoever to the structure of the trajectory itself. Where compaction happens is determined purely by step count or token count, not by "how far along the current line of reasoning is." The result: a search or a stretch of reasoning may be in a mid-state, yet gets cut off by compaction, some results are lost, and subsequent steps have to be redone or even go astray.

In other words, the fixed threshold treats compaction as a "resource management" problem, when it is fundamentally a "semantic decision" problem.

Anchoring Effect was originally a behavioral economics concept, referring to people over-relying on the first information they obtain (the anchor) when making decisions, with subsequent judgments being "pulled" toward one side by that anchor. In the long-horizon agent setting, it manifests as autoregressive ossification: erroneous reasoning the model has already written down, stale tool return values, and overturned assumptions all remain in context as tokens, continuously influencing the conditional probability distribution of every subsequent step. The model tends to "continue writing" along its existing statements rather than re-evaluating whether the premises hold. This is related to the LLM's lack of dynamic working memory—it cannot proactively "strike through" a discarded line of thought the way a human can, and can only use new tokens to overwrite old ones, while the old tokens still carry weight in the attention mechanism. Compaction matters precisely because it must proactively clear out these residual anchors, giving subsequent reasoning a clean starting point again.

SelfCompact's Approach: Handing Compaction Authority Back to the Model

SelfCompact's core claim is that the timing and granularity of compaction should be judged by the model itself during reasoning.

It consists of two paired inference-time components. One is the compaction tool: the model proactively calls it when needed to summarize the accumulated context, rather than waiting for an external scheduler to force execution at fixed points. The other is a companion judgment mechanism that lets the model decide, based on the current trajectory state, whether to invoke this tool.

This design implies a judgment: the model knows better than an external scaffold whether it is "currently halfway through reasoning, or already wrapping up." A fixed threshold can only see token counts, not semantic boundaries; whereas the model, during generation, has more direct information about whether the current step can be safely aborted and which intermediate results must be preserved.

From an engineering perspective, this is a "self-managed" rather than "externally managed" context strategy. Compaction turns from an external, content-agnostic timer into a callable action inside the trajectory.

Scaffold refers to the engineering framework wrapped around a base model, responsible for prompt templates, the tool-calling loop, memory management, retries, and termination conditions. It does not modify model weights; it only "steers" model behavior through scheduling logic. Common agent scaffolds include ReAct, Reflexion, and AutoGPT-style loop controllers. Fixed-threshold compaction is a typical hardcoded strategy at the scaffold layer: count tokens, and trigger summarization once you exceed K. What makes SelfCompact radical is that it transfers this control from the scaffold to the model's own tool-calling capability—the model proactively requests compaction via function calling, and the scaffold is only responsible for executing it. This means context management shifts from "the responsibility of an external scheduler" to "an action of the model," sitting at the same decision level as actions like searching or writing code, and thus can be planned together with other reasoning steps. This paradigm shift also echoes the recent trend in agent research of scaffolds gradually "thinning out" and pushing more control down to the model.

Why This Is Worth Watching

Making compaction a model-callable tool means context management is no longer hardcoded scaffold logic, but can adapt to the task type. For tasks with vastly different structures, such as multi-step search and long-chain reasoning, fixed intervals easily hit pitfalls, whereas the model's autonomous judgment can, at least in theory, align with task structure.

The cost is also straightforward: the model incurs extra judgment overhead, and its judgment itself may be wrong. Fixed thresholds are crude, but their behavior is predictable; self-management trades away some determinism for flexibility. Whether this trade-off is worthwhile depends on how sensitive the task is to "the integrity of intermediate results."

Currently, the information given in the original source stops at the methodological framework, lacking experimental data and failure case analysis. This limits assessment of SelfCompact's actual benefits—whether "the model decides for itself" holds up in a paper, and whether it remains stable in real long trajectories, still requires more evidence.

Context Window refers to the maximum number of tokens a model can hold in a single inference pass; beyond that it is truncated or errors out. Even though modern model windows have reached hundreds of thousands or even millions of tokens, long-horizon agents still hit the wall, and the reason is not just capacity: attention cost grows with sequence length (typically close to O(n²)), and long contexts also give rise to the "lost-in-the-middle" phenomenon—information located in the middle section sees a significant drop in utilization. Therefore, compaction is not just about saving tokens; it serves to maintain reasoning quality. The key to judging whether compaction is "safe" is the semantic boundary: a complete search, a closed proof, a confirmed intermediate conclusion—these are the smallest units that can be safely folded; whereas splitting apart an in-progress line of reasoning causes structural damage. A fixed threshold sees only token counts and cannot recognize these boundaries. This is precisely the core reason SelfCompact advocates for autonomous model judgment.

Summary

SelfCompact targets a real pain point in long-horizon agents: fixed-threshold compaction ignores trajectory structure and may discard partial results mid-reasoning. The solution is to hand the compaction tool to the model for autonomous invocation, replacing token-count judgments with semantic judgments. The direction is clear, but the public information is still insufficient to judge its stability and net benefits.

One-line Summary

SelfCompact lets the model autonomously decide when to compact context, replacing fixed-threshold strategies that ignore trajectory structure.

Paragraph Summary

A long-horizon agent's historical trajectory, mixed with stale content, produces an "anchoring effect" that keeps pulling subsequent reasoning in the wrong direction, while also eventually exceeding the context window. Existing scaffolds rely on fixed step counts or token thresholds to trigger summarization, a mechanism that ignores the semantic structure of the trajectory and may truncate in the middle of reasoning or search, losing intermediate results. SelfCompact proposes handing compaction authority back to the model: in the form of a "compaction tool" that the model proactively calls when needed to summarize the accumulated context, with a companion judgment mechanism deciding when to invoke it. This amounts to turning context management from external hardcoded logic into a callable action inside the trajectory, theoretically aligning with task structure. The cost is added judgment overhead and reduced behavioral determinism, and the public information includes only the methodological framework, lacking experiments and failure cases, so its actual benefits remain to be validated.