A New Approach to Federated Class Incremental Learning: Using Pseudo Features to Overcome Catastrophic Forgetting

FCLPF uses class prototypes to generate pseudo features, mitigating catastrophic forgetting in federated class incremental learning without storing old samples.
This article introduces FCLPF, a federated class incremental learning method aimed at solving catastrophic forgetting caused by continuously arriving data in federated learning. Traditional replay methods require storing old samples or training GANs to generate pseudo-samples, which bring privacy leakage and high computational costs, respectively. FCLPF maintains class prototypes (feature means) on each client and generates pseudo features accordingly to mix with new data for training, thereby mitigating forgetting with lightweight computation while protecting privacy. Experiments on CIFAR-100 and TinyImageNet show that FCLPF significantly outperforms baselines in accuracy and greatly reduces the forgetting rate (from 24.46% to 3.14% on TinyImageNet). The article concludes by discussing the limitations of the method in complex modalities and non-IID scenarios and future directions.
Federated learning typically assumes that data is fixed before training begins, but in real-world scenarios data often arrives continuously. This causes the model to forget old knowledge when learning new classes, a phenomenon known as catastrophic forgetting. To address this problem, researchers have proposed FCLPF (Federated Class Incremental Learning with Pseudo Features), a method that leverages prototypes to generate pseudo features and thereby mitigate forgetting.

Catastrophic Forgetting: The Roadblock to Deploying Federated Learning
Federated learning allows multiple clients to collaboratively train a model without sharing raw data, but conventional methods assume that each client's data distribution is static. In real applications, however, data often arrives as a stream—for example, new user behaviors, new device types, or new content categories. If the model is updated directly on new data, it will severely forget the old classes it previously learned.
One intuitive solution is to store exemplars of old data and replay them when training on new tasks. But this approach carries privacy leakage risks and incurs high storage costs, contradicting the original intent of federated learning—that "data never leaves the local device." Another idea is to use generative adversarial networks (GANs) to generate pseudo-samples of old data, but GAN training is complex and computationally expensive, making it difficult to deploy on resource-constrained clients.
FCLPF takes a different path: it generates pseudo features through class prototypes, requiring no storage of any real samples and avoiding the heavy computational burden of GANs.
Catastrophic Forgetting is a classic problem faced by neural networks in incremental learning: when a model is trained on a new task or new data, its parameter updates overwrite the knowledge learned from previous tasks, causing a sharp decline in performance on old tasks. This phenomenon was first systematically described by McCloskey and Cohen in 1989 and later studied in depth by French et al. in the 1990s. In class incremental learning scenarios, the model must learn new classes that appear sequentially while data from old classes is no longer available, making forgetting particularly severe. Common mitigation strategies include regularization methods (such as EWC, which constrains changes to important parameters), replay methods (storing or generating old data), and parameter isolation methods (assigning independent subnetworks to different tasks). FCLPF belongs to the pseudo-feature generation branch of replay methods.
The Core of FCLPF: Prototype-Driven Pseudo-Feature Generation
The core idea of FCLPF is to maintain a set of class prototypes locally on each client—that is, the mean vector of each class in the feature space. When new class data arrives, the client not only trains the model on the new data but also uses the stored prototypes to generate pseudo features that simulate the feature distribution of old classes, thereby balancing old and new knowledge during training.
Specifically, during local training, the client mixes pseudo features with real new data and constrains the model's output on old classes through a loss function, thereby mitigating forgetting. Since pseudo features are generated directly from prototypes, the computation is far less than training a GAN, and only prototypes or model parameters need to be transmitted, significantly reducing communication costs.
This design allows FCLPF to achieve a good balance among privacy protection, computational efficiency, and model performance.
Class Prototype refers to the mean of all sample feature vectors of a given class in the feature space. This concept originates from prototype learning and metric learning, whose core assumption is that samples of the same class cluster around the mean in the representation space, so the mean can serve as a compact representative of that class. In incremental learning, a prototype requires storing only a single vector rather than a large number of samples, resulting in minimal storage overhead and no leakage of raw data. When FCLPF uses prototypes to generate pseudo features, it typically adds Gaussian noise centered on the prototype or samples through a linear transformation to simulate the true distribution of old-class features. Prototype quality depends on the stability of the feature extractor; if the feature space drifts as new tasks are updated, old prototypes may become invalid, so maintaining prototype consistency is a key challenge for this class of methods.
Experimental Validation: Dual Advantages in Accuracy and Forgetting Rate
The researchers evaluated FCLPF on two image classification benchmarks: CIFAR-100 and TinyImageNet. On CIFAR-100, FCLPF achieved an average accuracy of 51.87% and an average forgetting rate of 9.62%, while existing baseline methods achieved an average accuracy of 47.72% and an average forgetting rate of 20.46%. On TinyImageNet, FCLPF reached an accuracy of 37.56% with a forgetting rate of only 3.14%, compared with 27.69% accuracy and 24.46% forgetting rate for the baseline methods.
The data show that FCLPF not only achieves higher accuracy but, more importantly, dramatically reduces the forgetting rate—on TinyImageNet, forgetting dropped from 24.46% to 3.14%, indicating that the model can better retain old knowledge while continuously learning new classes. This advantage is particularly critical in class incremental scenarios, because practical applications often require the model to reliably handle historical classes even after updates.
Average Accuracy and Average Forgetting are the two most commonly used evaluation metrics in class incremental learning. Average accuracy refers to the mean of the model's final accuracy across all learned tasks, measuring overall performance; average forgetting calculates the mean of the difference between the accuracy when the model finished learning each task and its final accuracy, measuring the degree of knowledge retention. The two typically exhibit a trade-off: the more the model favors new tasks, the higher the accuracy may be, but the more severe the forgetting. FCLPF reduced the forgetting rate on TinyImageNet from 24.46% to 3.14%, demonstrating that its pseudo-feature replay mechanism effectively constrains the damage that parameter updates cause to the decision boundaries of old classes, while the simultaneous improvement in accuracy indicates that old and new knowledge are well coordinated.
Implications and Outlook
FCLPF demonstrates the potential of pseudo-feature methods in federated class incremental learning. It avoids the privacy risks of storing real samples and bypasses the high computational cost of GANs, offering a lightweight solution for resource-constrained federated learning environments.
However, the method has so far been validated only on image classification tasks; its effectiveness on more complex modalities (such as text and speech) or in federated scenarios with a higher degree of non-IID data remains to be explored. In addition, the quality of the prototypes themselves directly affects the fidelity of the pseudo features, and how to maintain reliable prototypes under class imbalance or data heterogeneity is a direction worth attention in the future.
Overall, FCLPF provides a pragmatic technical path for deploying federated learning in dynamic data environments, and its "prototype + pseudo-feature" idea may also inspire more research that does not require replaying old data.
One-line Summary
FCLPF uses class prototypes to generate pseudo features, mitigating catastrophic forgetting in federated class incremental learning without storing old samples.
Paragraph Summary
This article introduces FCLPF, a federated class incremental learning method aimed at solving catastrophic forgetting caused by continuously arriving data in federated learning. Traditional replay methods require storing old samples or training GANs to generate pseudo-samples, which bring privacy leakage and high computational costs, respectively. FCLPF maintains class prototypes (feature means) on each client and generates pseudo features accordingly to mix with new data for training, thereby mitigating forgetting with lightweight computation while protecting privacy. Experiments on CIFAR-100 and TinyImageNet show that FCLPF significantly outperforms baselines in accuracy and greatly reduces the forgetting rate (from 24.46% to 3.14% on TinyImageNet). The article concludes by discussing the limitations of the method in complex modalities and non-IID scenarios and future directions.
Related articles

RePlan-VLA: Progress-Aware Replanning Breakthrough for Long-Horizon Robotic Manipulation
The RePlan-VLA framework converts VLA models' short-horizon capabilities into reliable long-horizon execution through progress-aware adaptive replanning, improving success rates by 21.9% and 10.4% on RoboCasa365 and CALVIN, and validating real-world deployment on a Franka Research 3 robot.

Geometric Bridge Flow Matching: A New Breakthrough in Blind Rigid Motion Correction
Geometric bridge flow matching enables blind rigid motion correction in head CBCT, achieving 37.1 dB PSNR and 0.27 mm RPE on simulated data in ~9.1 minutes, outperforming existing methods.

Google Releases Gemini 4 Hydrogen: 2.5x Faster Without Sacrificing Intelligence
Google releases the lightweight Gemini 4 Hydrogen: 2.5x faster task completion, AA Intelligence Index of 54 beating its flagship sibling, 7.3s single-step on Agent Arena faster than Claude and GPT rivals, four thinking levels with pay-as-you-need pricing, now available across AI Studio and more.