AdaExploreBench: A Benchmark for Proactive Exploration and Environmental Adaptation in LLM Agents

Large language model (LLM) agents are increasingly being deployed in unfamiliar and dynamically changing environments, where they need to proactively explore, understand, and adapt to environmental dynamics. However, existing benchmarks rarely test whether agents can proactively discover how an environment works and adjust when the environment evolves. To address this gap, researchers introduce AdaExploreBench—a benchmark for evaluating proactive exploration and test-time adaptation in evolving environments.

Why Is a New Benchmark Needed?

Current LLM agents perform impressively on tasks such as tool calling and multi-step reasoning, but these tasks often assume that environmental rules are static and known. In real-world scenarios, environments may contain undisclosed hidden rules, and the rules may even drift over time. AdaExploreBench instantiates environmental dynamics as hidden rules, requiring agents to proactively gather evidence and infer the rules. Under implicit rule drift, the underlying rules change without notice, and the agent must detect from degraded performance that its prior knowledge is outdated and re-explore.

Benchmark Design: Hidden Rules and Rule Drift

The core design of AdaExploreBench revolves around two challenges:

• Proactive evidence acquisition: Agents cannot passively wait for information; they need to design exploration strategies to collect sufficient evidence to infer the hidden rules.
• Implicit rule drift detection: When environmental rules change without announcement, agents need to recognize from degraded task performance that old knowledge is no longer valid and trigger re-exploration.

This design is closer to real deployment scenarios, such as API behavior changes, shifts in user preferences, or system policy updates.

Calibrated accuracy is a key metric for evaluating the quality of an agent's decisions under uncertainty. It examines not only whether the agent's final answer is correct, but also whether its confidence matches the true probability of correctness—an agent that always gives high confidence but frequently errs will have low calibrated accuracy. In hidden-rule scenarios such as those in AdaExploreBench, agents need to progressively update their posterior beliefs about the rules during exploration. Calibrated accuracy effectively reflects whether they "know what they know," thereby distinguishing blind guessing from reliable evidence-based inference.

Rule drift refers to the phenomenon in which the underlying decision rules of an environment change over time in a non-explicit manner. Similar to concept drift, it requires learning systems to detect distribution changes and trigger model updates. In the context of LLM agents, the difficulty of rule drift lies in the fact that the changes are implicit: with no notifications or explicit signals, agents can only infer from indirect clues such as a decline in task success rate that old rules have become invalid. This involves change detection and forgetting mechanisms—agents must proactively downweight outdated knowledge and re-explore to establish new rules.

Performance of Eight Frontier Models

The experiments covered eight leading LLMs. The results show that proactive and adaptive exploration remains challenging: GPT-6 Sol achieves only 61.9% calibrated accuracy. When sufficient evidence is provided or rule drift is removed, average performance improves significantly by nearly 10%. This contrast reveals a key bottleneck: current LLM agents can often reason effectively once they obtain valid evidence, but they struggle to proactively acquire the right evidence and to recognize when previously acquired evidence is no longer valid.

This finding corroborates recent discussions about agents being "strong at passive reasoning but weak at proactive exploration." According to the analysis in the benchmark paper, the performance gains mainly come from improved evidence availability rather than differences in model reasoning ability, indicating that the bottleneck lies more in exploration strategies and metacognition.

Implications for Agent Development

The results of AdaExploreBench suggest that developers building agents for dynamic environments should pay attention to the following capabilities:

1. Explicit design of exploration strategies: Do not assume that agents will naturally learn to explore efficiently; it may be necessary to introduce intrinsic reward or information-gain-driven exploration mechanisms.
2. Drift detection and forgetting mechanisms: Agents need to monitor their own performance and proactively discard outdated knowledge when performance degradation is detected.
3. Training signals for test-time adaptation: How to continuously learn during deployment without catastrophic forgetting remains an open problem.

This benchmark provides a standardized testbed for evaluating and diagnosing these capabilities and is expected to advance research on more robust LLM agents.

Intrinsic reward is a classic mechanism in reinforcement learning that drives exploration. It refers to reward signals generated internally by the agent rather than coming from an external task reward in the environment, with common forms including novelty rewards, prediction-error rewards, and information-gain rewards. In LLM agent exploration scenarios, intrinsic rewards can encourage agents to visit informative states or take actions that reduce uncertainty about the rules, thereby mitigating the tendency to "passively wait for evidence."

Catastrophic forgetting refers to the phenomenon in which neural networks rapidly lose previously learned knowledge when learning new tasks or new data. In test-time adaptation scenarios, agents need to continuously update their behavior based on new evidence during deployment while retaining old rules that remain valid. If updates are too aggressive, rules that still hold may be overwritten along with the rest; if too conservative, the agent cannot adapt to drift. This trade-off is one of the important causes of the bottleneck revealed by AdaExploreBench.

One-line Summary

The AdaExploreBench benchmark reveals the bottleneck that LLM agents are weak at proactive exploration and struggle to detect implicit rule drift.

Paragraph Summary

AdaExploreBench is a new benchmark for evaluating proactive exploration and test-time adaptation of LLM agents in dynamic environments. It instantiates environmental dynamics as hidden rules, requiring agents to proactively gather evidence to infer the rules and, when the rules drift silently, to detect from performance degradation that prior knowledge is invalid and trigger re-exploration. The experiments cover eight frontier models; GPT-6 Sol achieves only 61.9% calibrated accuracy. Providing sufficient evidence or removing rule drift improves average performance by nearly 10%, indicating that the bottleneck lies in proactive evidence acquisition and drift detection rather than reasoning ability. The article recommends that developers explicitly design exploration strategies, introduce drift detection and forgetting mechanisms, and explore training signals for test-time adaptation.