Visualizing Catastrophic Forgetting: How Replay and EWC Save Continual Learning

What Is Catastrophic Forgetting

When a neural network first learns Task A and then moves on to learn Task B, it tends to completely forget what it learned before — this is the well-known phenomenon of "catastrophic forgetting" in machine learning. The concept isn't hard to describe in a textbook, but helping students truly understand why it happens and what various mitigation strategies actually do is far from straightforward.

An educator teaching a continual learning course shared his experience on Reddit: students grasped the internal mechanisms of different methods much more quickly after seeing a simple visualization. He trained a small network on two different 2D point-sorting tasks and plotted 2D slices of the loss landscape in weight space. By watching the network "descend" through the loss landscape, forgetting becomes immediately apparent.

Round 1: Catastrophic Forgetting Under Naive Training

The first round used the most straightforward training approach. The network was first trained on Task A until it reached 100% accuracy, then training continued using only Task B data.

The core of the problem: from a loss landscape perspective, for the network to "go downhill" on Task B (reducing B's loss), it often means "going uphill" on Task A (increasing A's loss). The optimal weight regions for the two tasks don't overlap, and gradient descent only focuses on the current task's loss. The result — Task A accuracy dropped from 100% to 64%, while Task B reached 100%.

These numbers concretely quantify the severity of catastrophic forgetting: without any constraints, the network sacrificed more than a third of its performance on the old task to learn the new one. The visualization showed students that this isn't the network "not trying hard enough" — it's a fundamental conflict between the optimization directions for the two tasks.

Round 2: Replay's Mitigation Effect

The second round introduced the classic strategy of experience replay. The network's starting point and architecture were identical; the only difference was that during Task B training, 20 out of 200 Task A sample points were retained and kept participating in training.

Keeping just 10% of the old data produced a remarkably significant effect. The network ultimately maintained 93% accuracy on Task A while also reaching 100% on Task B. Compared to the first round's drop from 100% to 64%, the degree of forgetting was substantially reduced.

The intuition behind this is clear: the presence of a small number of old samples acts as a "constraining force" for Task A in the loss landscape. As optimization moves toward Task B, these replayed samples pull the weights toward a region that satisfies both A and B simultaneously, preventing the network from sliding entirely into a valley favorable only to B. Of course, with a limited number of replayed samples, slight forgetting still remains.

The idea of experience replay originally came from reinforcement learning — DeepMind used it in their 2015 DQN paper to break temporal correlations between training samples. Adapted for continual learning, it has evolved into several variants: the simplest is random storage of old samples (the approach used here); more complex is "Generative Replay," which trains a generative model to synthesize pseudo-samples from old tasks, completely bypassing the need to store original data — particularly important in privacy-sensitive settings like healthcare or finance. There is also "prioritized replay," which selects samples based on their representativeness or difficulty for old tasks rather than random sampling. The fundamental limitation of replay strategies is the "stability-plasticity dilemma": the higher the proportion of retained old samples, the better the old task is protected, but the less efficiently the network learns new tasks, and storage costs accumulate continuously over long task sequences.

Round 3: EWC Regularization Achieves Zero Forgetting

The third round used a regularization-based approach — Elastic Weight Consolidation (EWC). The most critical aspect of this round: no Task A data was used whatsoever.

EWC adds a penalty term to the loss function. When a weight that contributes significantly to Task A accuracy tries to deviate from the value it held after Task A training, the penalty increases. In other words, EWC identifies which weights are "critical" to the old task and places elastic constraints on them, while allowing less important weights to freely adjust to accommodate the new task.

The results were impressive: Task A accuracy briefly dipped to 97% during training before recovering, ultimately stabilizing at 100%, with Task B also reaching 100%. That is, without retaining any old data, EWC achieved the ideal outcome of no persistent forgetting.

EWC was proposed by Kirkpatrick et al. in 2017, with theoretical foundations rooted in Bayesian inference. It uses the Fisher Information Matrix to estimate the "importance" of each weight to the old task — Fisher information essentially measures how much the model's output distribution on the old task would change if a given weight shifted slightly. The larger the change, the more critical that weight, and the higher the corresponding penalty coefficient. The regularization term takes the form λ/2 × Σ Fᵢ(θᵢ - θᵢ)², where θᵢ is the weight value after training on the old task, Fᵢ is the corresponding Fisher information, and λ controls overall constraint strength. This design allows EWC to assign differentiated "elasticity" to different weights: weights irrelevant to the old task are barely constrained and can freely adapt to the new task, while critical weights are locked near their old values. This mechanism is conceptually analogous to the hypothesis of selective synaptic stabilization in the brain — which is also the origin of the name "Elastic Weight Consolidation."

Comparing Three Methods and Key Takeaways

Putting all three rounds together, the core tension and solution landscape of continual learning becomes clear:

Method	Uses Old Data?	Task A Final Accuracy	Task B Final Accuracy
Naive Training	No	64%	100%
Replay	Retains 10% of old samples	93%	100%
EWC Regularization	None at all	100%	100%

This comparison reveals two distinctly different technical approaches: replay combats forgetting through "re-exposure" to old data — direct and effective, but requires storing old samples, which may not be feasible in privacy-sensitive or storage-constrained scenarios; EWC protects old knowledge by constraining the movement of critical weights, requires no storage of original data, and in this small-scale experiment even achieved superior results.

What's truly worth emphasizing is the educational value of the visualization itself. The 2D slices of the loss landscape transform the abstract notion of "forgetting" into a visible "going downhill and uphill," letting learners intuitively understand that the root cause of forgetting is a conflict in optimization directions — and that different mitigation methods are fundamentally all about reshaping or constraining this loss landscape. For those new to continual learning, this kind of intuition often arrives faster and sticks longer than working through equations.

Of course, this is a simplified 2D toy experiment. In real large-scale models, task conflicts and weight importance estimation are far more complex. But as an introductory demonstration for understanding catastrophic forgetting and its mitigation mechanisms, it clearly presents the causal relationship between the problem and its solutions.

The continual learning field typically categorizes methods for mitigating catastrophic forgetting into three broad classes: regularization-based (e.g., EWC, SI, LwF), replay-based (e.g., experience replay, generative replay), and architecture-based (e.g., Progressive Neural Networks, PackNet), each with its own applicable boundaries. It's important to note that this experiment uses an extremely small network with only two tasks. While EWC achieves ideal zero-forgetting here, in practice, as the number of tasks grows, EWC's Fisher Information Matrix computation and storage costs scale linearly, and interference between accumulated constraint terms can cause performance degradation. Replay methods face similar challenges with storage capacity and sample selection strategies at scale. Real systems often need to combine multiple strategies rather than relying on any single approach.