Farewell to Replay Buffers: How TA-KAN Solves the OOD Detection Challenge in Continual Learning

Continual Learning enables models to learn new tasks sequentially like humans without forgetting old knowledge, but there is a long-overlooked weakness: when the model encounters inputs it has never seen before, can it still reliably say "I don't recognize this"? This is exactly the problem that Out-of-Distribution Detection (OOD) aims to solve. A recent paper proposes a framework called TA-KAN, which attempts to do two things well simultaneously in a rehearsal-free continual learning setting—not forgetting old tasks, and accurately identifying unknown samples.

Why OOD Detection Is Especially Difficult in Continual Learning

In static training scenarios, OOD detection already has relatively mature solutions: the model assigns confidence to inputs, and low confidence indicates an unknown sample. But continual learning breaks this premise. The model is trained sequentially on a task sequence, and training data from past tasks is typically no longer retained. Standard confidence scores shift systematically as tasks progress—the model becomes overconfident on inputs from recent tasks, while giving confused confidence on old task inputs and truly unknown inputs.

The paper summarizes the core of this problem into two constraints: (i) learning a task sequence without catastrophic forgetting; (ii) identifying OOD samples without access to past task ID data. Most existing methods rely on rehearsal buffers to store old data or assume a closed-world setting, which is limited by storage and privacy constraints in real deployment and difficult to scale with the number of tasks.

Catastrophic Forgetting refers to the phenomenon where, when a neural network learns a new task, parameter updates largely overwrite the weights that old tasks depend on, causing a sharp decline in old task performance. This is particularly pronounced in continual learning settings where the model is trained on a task sequence: the model never has the opportunity to see all data simultaneously, and the gradient direction only serves the current task. The academic community's approaches can be broadly categorized into three types—regularization methods (such as EWC constraining drift of important parameters), rehearsal methods (storing or generating old samples to mix into training), and architectural methods (allocating independent subnetworks or isolating parameters for tasks). The KAN backbone that TA-KAN adopts belongs to a variant of the third category, reducing parameter interference between old and new tasks through structural locality.

Recency Bias is another typical phenomenon in continual learning: because the model repeatedly encounters the latest task's data at the end of training, its decision boundaries and output confidence tilt toward recent tasks. In terms of OOD detection, this manifests as the model giving abnormally high confidence to inputs from the current task distribution, while the confidence distributions for old task samples or truly unknown samples overlap, causing threshold-based discrimination to fail. This shift is systematic and accumulates with the task sequence, and cannot be corrected by simple temperature scaling or post-processing, so it needs to be controlled at the classifier structure level.

TA-KAN's Two Key Designs

Using KAN's Local Plasticity to Combat Forgetting

The paper draws inspiration from Kolmogorov-Arnold Networks (KAN). Unlike traditional MLPs that place fixed activation functions on nodes, KAN places learnable functions on edges. This structure inherently possesses local neuroplasticity—when learning new tasks, perturbations to existing parameters are more localized, thereby alleviating catastrophic forgetting. The authors believe this makes KAN a suitable backbone for rehearsal-free continual learning.

Kolmogorov-Arnold Networks (KAN) is a neural network architecture proposed in 2024, based on the Kolmogorov-Arnold representation theorem: a multivariate continuous function can be represented as a composition of a finite number of univariate continuous functions and addition. Unlike MLPs that place fixed nonlinear activation functions on nodes and weights on edges, KAN places learnable spline functions (typically B-splines) on edges, with nodes only performing summation. This design brings two characteristics: first, higher parameter efficiency—expressing the same function typically requires fewer parameters; second, stronger interpretability and locality of functions, because the spline on each edge only takes effect locally within its domain. It is precisely this locality that makes KAN's modifications to existing functions more concentrated when learning new tasks, less likely to affect parameter regions related to old tasks, thereby providing a structural foundation for rehearsal-free continual learning.

Task-Adaptive Classifier: Suppressing "Recency Bias"

KAN alone is not enough. The paper points out that continual learning models generally exhibit recency bias, i.e., overconfidence on the last-learned task, which directly contaminates OOD scores. To address this, the authors design a task-adaptive classifier TA-KAN, which enhances the separability of ID and OOD samples by controlling activation locality on a per-task basis.

Geometry-Guided OOD Score

Beyond classifier confidence, the paper also proposes a geometry-guided OOD score as a supplement to TA-KAN's confidence. The key point is that this score does not depend on any past ID data, making it compatible with the rehearsal-free setting.

Results and Significance

The paper reports that the method significantly improves OOD detection performance while maintaining strong ID classification performance, achieving state-of-the-art results on the OOD-CL benchmark. Compared to approaches relying on rehearsal buffers, the value of this line of thinking lies in building "scalability" into the method itself: the more tasks there are, the heavier the storage and privacy burden of rehearsal buffer solutions, while rehearsal-free frameworks are not subject to this constraint.

To be objective, the abstract does not disclose specific benchmark datasets, task sequence lengths, or the list of comparison methods, and the stability of KAN-type backbones on larger-scale task sequences still requires more experimental validation. For teams focused on deploying continual learning, the inspiration of this work lies in: OOD detection should not be an add-on module after training is complete, but should be designed together with the architectural choices of continual learning.

Frequently Asked Questions

Does TA-KAN need to store old task data? No. The method is explicitly designed for the rehearsal-free setting, and the OOD score does not depend on past ID data.

Why not use ordinary confidence for OOD detection? In continual learning, confidence is affected by recency bias and shifts systematically, making judgments on old tasks and unknown samples unreliable.

What role does KAN play here? Its local neuroplasticity makes new task learning interfere with old parameters more locally, thereby mitigating catastrophic forgetting.

One-line Summary

TA-KAN uses a KAN backbone and task-adaptive classifier to balance anti-forgetting and OOD detection in rehearsal-free continual learning.

Paragraph Summary

The article introduces a paper addressing the OOD detection weakness in continual learning, proposing the TA-KAN framework. It first notes that OOD detection is especially difficult in continual learning scenarios: models are trained on task sequences and old data is no longer retained, and standard confidence shifts systematically due to recency bias. The paper summarizes this into two constraints: avoiding catastrophic forgetting and operating without historical ID data. It then introduces TA-KAN's two key designs: leveraging KAN's local neuroplasticity to reduce forgetting, and controlling activation locality through a task-adaptive classifier to suppress recency bias; supplemented by a geometry-guided OOD score that does not rely on historical ID data. Results show it achieves state-of-the-art on the OOD-CL benchmark while maintaining ID classification performance, though the abstract does not disclose specific datasets or comparison details.