EvoArena: Evaluating AI Agents' Continuous Adaptation in Dynamic Environments

When software versions update, terminal workflows shift, or user preferences continuously evolve, can AI agents timely update their knowledge and strategies — rather than relying on outdated experience? This is the core question that EvoArena, proposed by a team led by NUS PhD student Xu Xundong, aims to answer. Presented at the NICE paper sharing session, this work reveals a blind spot that existing agent benchmarks have long overlooked: the evolution of the environment itself.

The Blind Spot of Static Evaluation: Environments Are Always Changing

Traditional large model evaluation focuses on a "single function call": input a query, output an answer, and assess whether the model understands the prompt and produces the correct result. But when large models act as agents interacting with external environments, things get considerably more complex.

The researchers point out that an agent's behavior is "the result of the joint effect of the model, tools, memory, and environment state." This means that even if the model itself hasn't changed, the tools and memory remain the same, and even the task instructions are unchanged — if the environment changes, the agent's final behavior will change accordingly.

The problem is that most mainstream benchmarks assume a static environment. Whether it's WebArena's fixed Docker web environment, SWE-Bench's fixed code repositories, or Gaia and TerminalBench — the environment remains unchanged throughout evaluation. In the real world, however, files, APIs, and user preferences are constantly evolving. EvoArena was created precisely to fill this evaluation gap.

The design logic behind mainstream agent benchmarks like WebArena and SWE-Bench is worth understanding in more detail. WebArena packages target websites (e.g., shopping, forums, code hosting) into fixed Docker images, ensuring web content is completely consistent across evaluations. SWE-Bench extracts Issue-PR pairs from real GitHub repositories, using fixed code snapshots as the test environment. The intent behind this "frozen environment" design is to ensure reproducibility and fair comparison — all models face identical environment states, making results comparable. However, this design implicitly assumes that the environment a model faces at deployment matches the evaluation environment. In practice, this assumption often breaks down: websites get redesigned, dependencies get upgraded, and user habits shift. It is precisely this crack in the assumption that motivates the research behind EvoArena.

Distinguishing Three Concepts: Why "Environment Evolution"?

Xu Xundong specifically clarifies three concepts that are easily conflated in the community. The first is "Dynamic Task" — as seen in some SWE-Bench variants — where "dynamic" means new problems are dynamically generated at evaluation time to prevent data contamination. The second is "Agent Evolution," or agent self-improvement, which focuses on whether an agent can continuously improve its own capabilities through interaction or reflection.

EvoArena is concerned with neither dynamic task generation nor agent self-evolution, but rather Environment Evolution — examining how existing agents perform when the environment is continuously changing.

A strong dynamic environment benchmark should test three capabilities: continuous environment evolution (multiple subtly different versions within the same setting), version adaptation ability (whether the agent can adjust its behavior after perceiving a change), and chain reliability (whether it can track a series of environmental changes and complete tasks consecutively, rather than handling each as an isolated task). In short: EvoArena tests whether an agent can detect environmental changes and adjust its behavior to adapt.

"Agent Evolution" (agent self-improvement) is a hot research direction at the intersection of reinforcement learning and large models. Representative works include enabling agents to iteratively improve themselves through self-play, reflection on failed trajectories, or synthetic training data (e.g., Voyager, SELF-REFINE). The core question in this line of research is "how can the agent become stronger," with the optimization target being the agent's own policy or parameters. EvoArena's motivation is entirely different: it does not modify the agent but instead changes the environment, observing whether existing agents can perceive changes in unfamiliar new environment states and adjust their behavior accordingly. This distinction is crucial to the experimental design — EvoArena evaluates an agent's "environment perception and adaptation ability," not its "self-improvement ability," and can therefore be used to fairly evaluate any existing agent framework without modifying its internal mechanisms.

Three Scenarios: From Terminal to Codebase to Personalized Dialogue

EvoArena builds dynamic environments across three scenarios that have attracted significant community interest.

Terminal-Bench Evolve: Workflow Scenario

Built on top of Terminal-Bench, the approach keeps the goal of each task unchanged while continuously modifying execution conditions. For example, the original task might be to deploy hello.html on port 8080 — then each subsequent evolution changes the execution conditions: adding hook and retry mechanisms, hardcoding the deployment path, modifying user or group permissions, changing paths, and so on. Each task thus spawns five environment states (environment 0 through 4), testing whether the agent can robustly complete the deployment goal across a series of changing execution conditions.

SWE-Chain Evolve: Software Engineering Scenario

Here, the environment is the code repository itself. Researchers collect data from a rich set of GitHub repositories, grouping related commits into "milestones" — and using milestones rather than individual commits as the unit of evolution. This is because in real repositories, a complete feature is often split across multiple consecutive commits; milestones better represent a single complete functional evolution.

Evaluation uses two metrics: fail-to-pass and pass-to-pass. The former measures whether the agent can implement the new functionality introduced in a new milestone (failing in the old version, passing in the new), while the latter measures whether the agent can implement new requirements without breaking functionality that already passed in the old version. This directly tests the agent's ability to track codebase evolution.

Persona-Mem Evolve: Social Intelligence Scenario

This is a long-context multi-turn dialogue scenario where the environment is the user's continuously evolving preferences in conversation. Researchers collect seed personas from Persona Hub, expand them into structured profiles, and then have a large model implicitly express preferences through dialogue — rather than stating them directly. For instance, a user might initially enjoy drinking coffee, then evolve to only drinking coffee on Monday mornings, then further evolve to drinking tea on weekday mornings and cappuccinos on weekends.

Researchers define preferences with over 70% semantic similarity as continuous evolution, concatenating them into long-context conversation histories while inserting unrelated topics as distractors. To make accurate recommendations, the agent must track the complete chain of preference changes, understanding the conditions, timing, and context under which each preference applies.

A Dismal Report Card: Even the Strongest Models Fall Short

Researchers tested with the strongest agent frameworks available at the time for each scenario (Terminus 2, OpenHands, Agentic Memory), paired with top-tier models including GPT-4.5, GLM-4.1, and Gemini 2.5 Pro. They evaluated on two dimensions: Step Level (each milestone evaluated independently) and Chain Level (the entire evolution chain must be completed correctly to pass).

The results are discouraging. Even GPT-4.5, which had the highest Step Level scores, barely exceeded 50% accuracy. Chain Level scores were even lower — the best-performing Gemini 2.5 Pro and GLM-4.1 reached only around 28–29%. Chain Level scores are significantly lower than Step Level because they require completing new requirements without breaking historical evolution, making the task inherently harder. This clearly demonstrates that even the strongest current agents perform poorly at tracking dynamic environments.

EvoMem: Git-like Patch Memory for Tracking Environmental Changes

To address this shortcoming, the team proposes EvoMem. An intuitive question arises: why not simply update the agent's memory? The answer lies in the fact that in dynamic environments, knowledge is "environment-dependent" — old information is useful in old environments, and new information is useful in new environments. Existing memory systems always overwrite old memories with the latest state; once the environment rolls back, the memory that was valid at that point has been overwritten, and performance naturally degrades.

EvoMem takes an approach similar to Git diff, capturing changes between different states of memory. It stores fields including the before and after states, a summary of changes, the reason for the change (inferred by the agent from interaction context), and the task that triggered the change.

When a new task arrives, the agent retrieves both from the original base memory and from the patch memory for historical change information, retaining more traces of environmental evolution in dynamic settings.

Main experiments show that all baseline agents improve after incorporating EvoMem, with Chain Level improvements notably larger than Step Level: the Terminal scenario sees a ~6 percentage point Chain Level improvement (only 2.4% at Step Level), the SWE scenario improves by 2 percentage points, and the Persona-Mem scenario improves by 3.2 percentage points. This demonstrates that EvoMem is particularly effective for tracking environmental changes and completing evolution tasks sequentially.

Interestingly, EvoMem is also effective on traditional "static" benchmarks like Gaia and Locomo. Researchers attribute this to the fact that in these long-horizon tasks, the agent's multi-turn interactions themselves change the environment state (e.g., calling a search engine introduces new information), so these can still be viewed as a form of environment evolution.

Git's diff mechanism is a key analogy for understanding EvoMem's design philosophy. In Git version control, each commit does not save a complete copy of the file but rather records a "patch" relative to the previous version — which lines were added, which were deleted. This saves storage while enabling precise restoration of any historical version or switching between versions. EvoMem applies the same idea to agent memory management: rather than overwriting old memories with new states when the environment changes (equivalent to keeping only the latest file copy), it explicitly records "what changed from state A to state B." This incremental memory structure allows the agent to access both current environment knowledge and trace historical changes — without losing context when it needs to handle an older environment version or understand the trajectory of change. This is also the fundamental reason EvoMem works naturally well in "environment rollback" scenarios.

In-Depth Analysis and Future Directions

Researchers provide a detailed breakdown of EvoMem's effectiveness across the three scenarios. In the Terminal scenario, accuracy is higher when patches are actually retrieved, used in the agent's trajectory, and applied in actual command-line operations. In the software engineering scenario, EvoMem significantly reduces the regression rate (breaking existing functionality) from around 9%. In the Persona-Mem scenario, EvoMem outperforms baseline memory on both coarse-grained (cross-level) and fine-grained (row-level) preference evidence retention.

One particularly thought-provoking finding concerns the relationship between token usage and accuracy: GPT-4.5 generates a large number of tokens, yet its accuracy does not necessarily surpass GLM-4.1 or Gemini 2.5 Pro, which generate fewer tokens. This hints that current scaling approaches have not truly solved the problem of dynamic environment tracking.

During the Q&A session, Xu Xundong also frankly acknowledged the method's limitations. As the number of interaction rounds increases and patches accumulate, efficient retrieval will become a bottleneck — he noted that pure semantic similarity retrieval may be insufficient, and that future work may need to model environment structure or draw on ideas like causal graph retrieval. For environment rollback, EvoMem is naturally effective since it stores inter-state changes; but when old and new rules conflict, the agent needs condition-based reasoning to determine which version of memory to invoke.

EvoMem remains a relatively general and simple method — it represents a first step toward improving agent performance in dynamic environments. The researchers believe that memory is an excellent interface for tracking dynamic environments, and that better capturing changes and going beyond semantic similarity for retrieval will be important directions worth pursuing.