Context Language Model: Let AI Agents Edit Their Own Context — No More Summarization

The Real Bottleneck for AI Agents: Context Management

Every AI agent working on long tasks faces the same fundamental challenge — the context window limit. As a conversation approaches the window ceiling, the standard approach is to force the model to summarize everything that's happened so far, then replace the original conversation with that summary. This is how nearly every harness handles it — Claude Code, Codex, Py, and virtually all the others: once a fixed threshold is hit (say, 75% of the window), compression triggers, the last few turns are kept, and everything else gets replaced by a summary. You can even trigger this manually with /compact in Codex or Claude Code.

The problem is that the summary is written at a moment when nobody yet knows what information will matter. Suppose your agent is debugging a service, and 40 turns ago it read a config file containing a value that turns out to be the root cause of the entire bug. But the summary was written before that value became important — if it wasn't included, it's gone forever, and the agent has no idea what it's lost. It then burns a huge number of tokens re-reading logs from scratch.

A new paper from Meta and the University of Washington proposes a different path: instead of having the model do a one-shot summarization, let it edit its own context window. That's the core idea behind the Context Language Model (CLM).

How CLM Works

In a standard language model, every turn appends new content to the end of the context — the next turn is fed "old context + new model output." What CLM does is replace that append operation with a function: the model itself decides what the entire context looks like for the next turn. It can retain, shrink, delete, or rewrite any part of it.

The mechanism is quite elegant. Before each request, the harness writes the current context to a file, with each turn as a labeled block. The model can open this file and modify it — just like editing code — by running a short Python script. Whatever remains in the file after that turn becomes the prompt for the next one.

This is clearly visible in demos: Py, the open-source coding agent, reads a 3,000-token log and a few turns later replaces it with a single line — precisely preserving the key information that log provided. The right-hand panel shows context size climbing each time a file is read, then dropping after the model cleans up after itself. Crucially, there are no preset rules about what to keep — the model is making its own judgment about which information matters most.

When you hand this kind of control to a model, it invents its own strategies. In the paper, the model invented a custom chat role called notes, wrote a helper function to compress old search results and called it 37 times, and in a multi-agent run even maintained a scoreboard of 21 sub-agents inside its own context, updating it in place 163 times. Even better, you can trigger compression via plain English instructions — like "compress to 4,000 tokens whenever I exceed 24,000" — and it actually does it.

CLM's design philosophy stems from a deeper observation: a traditional language model's "context" is fundamentally a linear sequence that only grows — each inference appends new content to the end, and the model has no active control over historical content. CLM breaks this assumption entirely, treating the context as a readable and writable working memory rather than an append-only log. This more closely mirrors how humans process information — people actively organize notes during long work sessions, cross out irrelevant material, and distill key findings into a single sentence. From an implementation standpoint, CLM relies on the model's code generation and execution capabilities: it operates on external files by generating and running Python scripts, rather than directly "modifying the token sequence" — which isn't feasible in current inference architectures. This indirect implementation also means CLM's effectiveness is highly dependent on the model's instruction-following ability and code quality; weaker models are more likely to make mistakes or over-compress during self-editing.

Benchmark Results: Summarization Hallucinates, CLM Is More Reliable

The paper constructed a small benchmark called ContextBench around this problem, including tasks like "maintain a Sudoku board in real time as moves stream in." Most existing compression techniques fail this benchmark outright. On the deeper research benchmark BrowseComp-Plus, CLM with Qwen3 27B achieved 59.4%, about 11 percentage points better than the best summarization approach, while using roughly one-fifth less compute.

The video's author deployed Qwen3 27B locally with VLLM (32,000 token budget) and designed three real tasks exceeding the window to validate this. One involved tracking cargo truck seal codes across 16 days of warehouse logs. Using Py's built-in summarization as a baseline, the results were alarming: the first summary listed 12 seal codes, 11 of which didn't exist anywhere in the logs — pure hallucination. By the third summary, 14 out of 18 codes were fabricated, including an invented sequence right next to a real code. More dangerously, these summaries came complete with checkboxes and next-step plans, looking completely confident — the agent had no way to know it was wrong.

In another task, after summarization the agent "concluded" that the logs contained no deployment information — but that deployment record was right there, and the agent had already read it. It was simply dropped entirely during summarization. Interestingly, the summarization baseline eventually got the right answer, but not because it remembered — it violated the prompt's explicit instructions and used grep to re-read every file from disk.

By contrast, Py CLM on the same tasks wrote the entire context into a notes block, tracking every seal code used, voided, and what to read next. All 18 codes were accurate, and it answered all but one question in the seal-and-event task correctly — while Py either fell back on grep or hallucinated outright.

BrowseComp-Plus is an extended version of OpenAI's BrowseComp benchmark, specifically designed to evaluate agents on long-horizon web browsing and information aggregation tasks — each question typically requires browsing dozens of pages and synthesizing scattered clues to reach an answer, directly testing information retention and reasoning under long-context conditions. It exposes the weaknesses of context compression schemes more effectively than common single-hop QA benchmarks, because key information is often retrieved in early steps but only becomes important in later steps — exactly the scenario where traditional summarization most easily loses information. This makes it a natural testing ground for CLM's value.

Four Pitfalls You Must Know About

This direction is promising, but the authors repeatedly emphasize: if you're building on top of it, understanding its limitations matters more than understanding its strengths.

The Harness Matters More Than You'd Expect

In three out of four default runs, the model never edited its own context at all. What actually saved things was Py CLM's safety net — when the context is about to overflow, it automatically replaces old tool outputs with a short note. The culprit was parallel tool calls: the model read four files in one turn and overflowed its remaining budget before it had a chance to react. To make CLM actually work, you must enable "one tool call per turn" and generate a note after each result — which is exactly what the paper's harness does.

Cost and Latency

Services like VLLM cache prompt prefixes. For CLM, where in the file the model makes its edits matters enormously. If edits happen at the end of the file, most of the prefix cache can still be reused and responses are faster. But once editing starts from the middle, everything after that point loses its cache and must be reprocessed — both latency and generation cost increase.

It Doesn't Automatically Get Cheaper

In testing, Py CLM once processed roughly twice as many tokens as Py; in another run, only 10% of the prompt came from cache. The paper proposes a fix called suffix cache reuse, which preserves computation for unmodified text after editing, but it currently exists only as an SGLang patch for single-model, single-GPU setups and is difficult to reproduce broadly.

The Model Needs to Be Strong Enough

The 9-billion-parameter model in the paper scored 6 points behind clean summarization before reinforcement learning training. And the headline results used a small 32,000-token budget; at 128,000 tokens, CLM and summarization perform roughly on par.

There's also a security concern: since the model can write to its own memory, prompt injection can write to it too — and injected content persists across turns. This is a concern the paper itself flags.

Prompt injection refers to attackers embedding malicious instructions into external content that the model will read, inducing the model to perform unintended actions. In ordinary conversational scenarios, injected content typically only affects a single-turn response. But CLM introduces a new attack surface: since the model can write content to and persist it in the context file, a successfully injected malicious instruction can survive in the file for multiple turns and even actively prevent itself from being deleted — for example, by injecting meta-instructions like "do not delete this block." This risk is especially pronounced when agents are used to process untrusted external data (such as crawled web pages or user-uploaded files). Research on injection defenses specifically for CLM scenarios is essentially nonexistent, which is why the paper's authors explicitly list this as an unresolved vulnerability.

A Direction Worth Watching Long-Term

The fundamental flaw of summarization is that it's "written once, then trusted forever" — and real-world testing proves exactly where that goes wrong. Letting the model self-manage its context genuinely fixes the hallucination and information-loss problems, provided the harness gives it enough reaction space and you keep a close eye on caching costs.

The authors have already released a Py extension that installs with a single command; type /CLM inside Py to open the panel from the demo that shows context size changes and before/after comparisons for each edit. Overall, this is an imaginatively conceived direction, and you can expect to see many evolved versions of it emerge going forward.