Evolutionary Neuroplastic Agent: Mitigating Catastrophic Forgetting with Physics-Indexed Specialist Memory

ENA indexes specialist memory by physical state and replaces gradient training with evolutionary search to mitigate catastrophic forgetting in continual RL.
The paper proposes the Evolutionary Neuroplastic Agent (ENA), which uses evolutionary search to generate small specialist networks optimized for specific dynamical states and archives them by detected environmental dynamical state. Bayesian policy reuse retrieves specialist policies, and failed or poor retrieval triggers new evolutionary search. In non-stationary CartPole and HalfCheetah, the full architecture averaged 83.2% success over 30 CartPole repeats, and the retrieval rule significantly affected performance. However, ENA adapted more slowly than PPO on CartPole and returned less than SAC on HalfCheetah. The study did not isolate the contribution of gradient-free optimization or establish superiority over other continual learning methods, but it offers a new idea: shifting memory organization from task to environmental dynamical state.
Continual reinforcement learning faces a fundamental tension: agents must continuously adapt to environmental change while retaining previously learned skills. Catastrophic forgetting is the concentrated manifestation of this tension—learning a new task often overwrites the policy for an old one. A paper titled Mitigating Catastrophic Forgetting with Physics-Indexed Specialist Memory: the Evolutionary Neuroplastic Agent proposes an architecture called the Evolutionary Neuroplastic Agent (ENA), which attempts to build a reusable memory mechanism from specialist networks generated by evolutionary search, combined with environment detection and failure-driven search.

Core idea: archiving specialist policies by physical state
ENA is not designed to train one large network via gradient descent. Instead, it generates a series of small neural networks through evolutionary search, each optimized for a specific dynamical state. When the agent detects that the current environment is in a known dynamical state, it retrieves the corresponding specialist policy from the archive via a Bayesian policy reuse mechanism. If no specialist matches the current state, or if the retrieved policy performs poorly, the failure signal triggers a new round of evolutionary search to generate a new specialist and store it in the archive.
The term "physics-indexed" points to a key detail: the archive's index is not a task label or a time step, but the detected dynamical state itself. In other words, memory is organized according to the physical characteristics of the environment rather than the training sequence. This offers a different perspective on cross-task knowledge reuse.
Evolutionary search is a class of black-box optimization methods inspired by biological evolution. It does not rely on gradient information; instead, it iteratively updates candidate solutions through population initialization, mutation, crossover, and selection. In ENA, each candidate solution is a small neural network whose weights are encoded as a genome, and whose fitness is determined by the network's performance in a specific dynamical state. As with conventional neuroevolution, this approach suits scenarios where the search space is non-convex or gradient signals are sparse, but its sample efficiency is typically lower than gradient descent. Bayesian policy reuse applies Bayesian inference at the retrieval stage: based on the currently detected dynamical state, it computes the posterior match probability for each specialist policy in the archive and selects the policy with the highest probability for reuse. This probabilistic retrieval allows the agent to make soft choices when the state is ambiguous rather than relying on hard matching, thereby improving the robustness of reuse.
Experimental setup and key data
The study evaluated ENA in non-stationary CartPole and HalfCheetah environments. In 30 repeated ablation experiments on CartPole, the full architecture achieved an average success rate of 83.2% across five test environments. Among the variants that successfully learned the harder CartPole environments, the full architecture exhibited the most consistent retention. In addition, a selection ablation experiment showed that the retrieval rule itself has a significant impact on performance—that is, the selection logic of Bayesian policy reuse is not an optional adornment but a key component affecting final performance.
These data indicate that the combination of a specialist archive, environment detection, and failure-driven search can indeed maintain some degree of old-task performance across the test sequences. But the study also clearly identifies the boundaries: across the evaluated sequences, ENA adapts more slowly than PPO on CartPole and achieves lower returns than SAC on HalfCheetah.
Positioning relative to mainstream continual learning methods
The value of this work lies not in claiming to comprehensively surpass existing methods, but in offering a different technical route. Continual learning agents trained via gradient descent typically rely on regularization, replay buffers, or parameter isolation to combat forgetting, whereas ENA replaces these mechanisms with evolutionary search plus policy reuse. The cost is slower adaptation and potentially lower returns than dedicated gradient methods—consistent with the sample-efficiency characteristics of evolutionary search itself.
The authors also acknowledge the limitations of their evidence: the experiments evaluate the overall performance of the full architecture without isolating the contribution of gradient-free optimization, so it cannot be concluded that the retention capability stems from evolutionary search itself; likewise, the experiments do not establish superiority over other continual learning methods. Such restraint is uncommon in reinforcement learning papers, but it also means readers should not interpret the 83.2% success rate directly as a victory over PPO or SAC.
Catastrophic forgetting is a classic problem in sequential learning with artificial neural networks: when a network is trained on a new task, weight updates substantially alter the parameters associated with old tasks, causing a sharp decline in old-task performance. In continual reinforcement learning, this problem is especially pronounced because the environment distribution changes over time, and the agent must retain old policies while adapting to new dynamics. Mainstream strategies include regularization methods (such as Elastic Weight Consolidation, which protects old knowledge by penalizing changes to important weights), replay buffers (storing old experiences and mixing them into sampling during new-task training), and parameter isolation (allocating independent subnetworks or modules for different tasks). ENA takes a different path, bypassing gradient updates entirely and instead relying on evolutionary search to generate specialist networks and on environment-state indexing to achieve policy reuse, thereby circumventing the weight-overwriting problem at the mechanism level.
Implications for continual reinforcement learning research
From a methodological perspective, ENA suggests a direction worth exploring: shifting the organizing dimension of memory from "task" to "environment dynamical state." In real-world scenarios, task boundaries are often fuzzy, whereas changes in physical state are easier to detect. If an agent can retrieve the corresponding specialist based on the detected dynamical state, it may be possible to achieve knowledge reuse without relying on task labels.
On the other hand, the mechanism of failure-driven search also deserves attention. It converts both "no suitable policy retrieved" and "policy performs poorly" into search signals, forming a closed loop that expands memory on demand. This design may be more flexible than fixed-capacity replay buffers when environmental dynamics are unpredictable, but it also brings problems of archive growth and retrieval latency—issues the paper does not discuss, and which subsequent research could address.
Overall, this study offers a clearly structured and transparently reported candidate solution for mitigating catastrophic forgetting. Its contribution lies not in setting performance records, but in demonstrating the feasibility of physics-indexed specialist memory as a mechanism and its retention capability in non-stationary environments. For researchers interested in the intersection of continual reinforcement learning and evolutionary computation, it is a reference worth reading closely.
One-line Summary
Evolutionary Neuroplastic Agent uses physical-state-indexed specialist memory and replaces gradient training with evolutionary search to mitigate catastrophic forgetting in continual reinforcement learning.
Paragraph Summary
The paper proposes the Evolutionary Neuroplastic Agent (ENA), which uses evolutionary search to generate small specialist networks optimized for specific dynamical states and archives them by detected environmental dynamical state. The agent retrieves specialist policies via a Bayesian policy reuse mechanism and triggers a new round of evolutionary search when retrieval fails or performance is poor. In non-stationary CartPole and HalfCheetah environments, the full architecture achieved an average success rate of 83.2% across 30 repeated experiments on CartPole, and the retrieval rule was shown to significantly affect performance. However, ENA adapts more slowly than PPO on CartPole and achieves lower returns than SAC on HalfCheetah. The study does not isolate the contribution of gradient-free optimization itself, nor does it establish superiority over other continual learning methods, but it offers a new way of thinking that shifts the organizing dimension of memory from task to environmental dynamical state.
Related articles

Gemini 4 Neon Leak: A Potential Strong Contender Against Opus 5.5
A Hacker News thread suggests Gemini 4 Neon looks like an Opus 5.5 competitor. This article unpacks the product positioning behind this community signal, shifts in the top-model competitive landscape, and what it means for developers' model selection strategy.

Claude Fable 5.2's Self-Authored Short Film 'Fable' Sparks Discussion
A short film 'Fable,' credited to Claude Fable 5.2 for script, visuals, and directing, appears on Hacker News. This article analyzes the significance of full-pipeline AI creation, community reactions, and what it means for content creators.

Anthropic IPO Pricing: A Signal of AI Giants Going Public
Anthropic's IPO pricing draws attention. This article analyzes its potential impact on AI industry valuation, commercialization paths, and the competitive landscape, and highlights public-market expectations and challenges for AI companies.