[KongchangAI]
· 2 min read· 1,217 words

CoReMem: Teaching Coding Agents to Pay for Their Memories

CoReMem: Teaching Coding Agents to Pay for Their Memories

CoReMem inversely values memory using downstream task performance, internalizing write and read costs to improve both solve rate and efficiency.

Aiming at the fundamental tension of cross-task memory in coding agents—high cost when too much memory is written, lost information when too little is written—the CoReMem framework proposes a solution that internalizes memory costs. It uses prospective segment valuation to compare the impact of a trajectory segment on the next task under the same repository state, quantifies memory value, and guides the construction of compact memory anchored in repository locations and executable checks. The actor adaptation mechanism estimates on-policy memory value within rollout groups, and the reward design prioritizes solve rate while penalizing steps in successful runs. Provenance-gated value credit assigns extra value only to steps that use memory anchors, avoiding free-riding. On ChainSWE, a 27B model's solve rate rises from 21.0% to 25.7%, steps decrease by 17%, and tokens decrease by 11%.

When a coding agent works on consecutive tasks in the same codebase, it carries its previous modifications and memories into the next task. But the agent doesn't know in advance what the next task will be. This creates a fundamental tension: writing too much memory might be useful later, but it also means higher write and read costs; writing too little risks missing critical information. How to strike the balance? The CoReMem framework offers an answer that internalizes the costs.

CoReMem paper source page

The value of memory needs to be "measured," not guessed

Traditional approaches tend to rely on heuristic rules to decide what to memorize—such as keeping recent modifications or prioritizing frequently occurring file paths. CoReMem's core idea is different: it quantifies the actual value of each trajectory segment for the next task through prospective segment valuation.

Specifically, researchers compare the performance difference on the next task under the same inherited repository state, with and without a given trajectory segment. This A/B-style comparison makes the value of memory measurable rather than a matter of intuition. Based on this, CoReMem guides the construction of a compact memory structure organized around anchors—which include repository locations and executable checks. This means memory is not a long natural-language summary, but an index pointing to key code locations and verification methods.

How the agent learns "when to use memory"

Having memory is one thing; solving another problem is also necessary: as the task progresses, the agent needs to judge when relying on existing memory can save effort. CoReMem's actor adaptation mechanism estimates on-policy memory value within rollout groups, comparing performance with and without memory.

Its reward design has a notable detail: it prioritizes problem-solving rate while penalizing the number of steps in successful runs. This guides the agent to pursue efficiency in solving problems rather than meaninglessly adding steps. In other words, the value of memory is reflected not only in "whether it can be solved" but also in "how fast it can be solved."

On-policy is a core concept in reinforcement learning, referring to training data (rollouts) generated by the policy currently being optimized, rather than from an old policy or human demonstrations. The reason CoReMem's actor adaptation emphasizes estimating memory value within rollout groups is that only by comparing "with memory" and "without memory" on on-policy data can the true marginal contribution of memory under the agent's current behavioral pattern be reflected. If off-policy data were used, the memory value estimate would be distorted by policy distribution shift, causing the reward signal to diverge from actual decisions. This also explains why the reward design must simultaneously penalize the number of steps in successful runs—it forces the agent to actively learn on-policy trajectories "when calling memory is more worthwhile" rather than passively relying on heuristic rules.

Provenance-gated credit assignment

In reinforcement learning, credit assignment has always been a challenge. CoReMem adopts provenance-gated value credit: the extra value term is assigned only to steps identified as having used anchors provided by memory, while each step still retains outcome credit.

This design avoids the "free-rider" problem—steps that did not use memory do not receive extra rewards due to the overall benefits brought by memory, making the learning signal more precisely point to effective memory-use behaviors.

Credit assignment is a classic challenge in multi-step decision-making reinforcement learning: when a trajectory ultimately receives a reward, how do we determine which step or which parts of the behavior contributed to success? In memory-augmented agents, the problem is more complex—overall gains may come from anchors provided by memory or from the agent's own reasoning ability. The essence of provenance gating is a structured mask: it uses traceable provenance markers in the trajectory (i.e., whether a step references a memory anchor) to precisely route the extra value term to the relevant steps. This is related to the ideas of eligibility traces or reward shaping, but the difference is that the gating is based on verifiable provenance information rather than temporal decay. Its direct effect is to reduce the variance of credit assignment, enabling policy gradients to more stably reinforce the behavioral pattern of "effectively using memory."

Empirical results: improvements in both solve rate and efficiency

On the ChainSWE benchmark, CoReMem raises the solve rate of a 27B-parameter actor model from 21.0% to 25.7%. At the same time, compared with an untrained actor model without memory, total steps are reduced by 17% and cache-aware total tokens are reduced by 11%—this already includes the overhead of memory module calls.

These data show that internalizing memory costs does not come at the expense of efficiency; instead, it improves the solve rate while reducing resource consumption. The absolute improvement in solve rate may not seem large, but in complex scenarios like consecutive coding tasks, every percentage point of progress corresponds to a large reduction in failure cases.

ChainSWE is a benchmark for consecutive software engineering tasks. Its core design is to have the agent process multiple interrelated tasks sequentially on the same codebase, with each task inheriting the repository state from the previous tasks. The key difference from traditional single-task SWE benchmarks is that the agent must manage state and memory across tasks, and the content of subsequent tasks is unpredictable to the agent. Therefore, ChainSWE is naturally suited to evaluating the value of memory mechanisms—writing too much memory increases costs, while writing too little loses critical context, and the benchmark amplifies the impact of this trade-off through the cumulative effect of task chains. CoReMem's improvement on this benchmark (+4.7 percentage points in solve rate) needs to be understood under this long-chain, high-uncertainty setting; its significance lies not only in the absolute value but also in simultaneously achieving reductions in steps and tokens.

Implications for coding agent design

CoReMem's contribution is not in proposing some larger memory module, but in establishing a framework that uses downstream task performance to inversely guide memory design. Memory is no longer static storage, but a dynamic component that can be valued, selected, and trained.

For teams building coding agents, this means that when considering memory solutions, write and read costs should be included in the optimization objective rather than just storing first and worrying later. The two mechanisms of prospective valuation and provenance-gated credit assignment can also be transferred to other agent scenarios that require cross-task state transfer.

One-line Summary

CoReMem uses downstream task performance to inversely value memory, internalizing write and read costs and achieving improvements in both solve rate and efficiency.

Paragraph Summary

Aiming at the fundamental tension of cross-task memory in coding agents—high cost when too much memory is written, lost information when too little is written—the CoReMem framework proposes a solution that internalizes memory costs. Through prospective segment valuation, it compares the impact of a trajectory segment on the next task under the same repository state, quantifies memory value, and guides the construction of compact memory anchored in repository locations and executable checks. The actor adaptation mechanism estimates on-policy memory value within rollout groups, and the reward design prioritizes solve rate while penalizing steps in successful runs. Provenance-gated value credit assigns extra value only to steps that use memory anchors, avoiding free-riding. On ChainSWE, a 27B model's solve rate rises from 21.0% to 25.7%, steps decrease by 17%, and tokens decrease by 11%.

#CoReMem#coding agents#memory cost#prospective segment valuation#provenance-gated credit assignment#ChainSWE#reinforcement learning
Share:

Related articles