Cross-User Adaptation for sEMG Gesture Recognition: Continual Learning Strategy Comparison and Training Stability

Introduction

Surface electromyography (sEMG) signals are becoming a core input for prosthetics and assistive interfaces because they directly reflect the user's muscle activity intent. However, sEMG signals are significantly affected by individual physiological differences, causing model performance to degrade in cross-user scenarios. Traditional recurrent neural networks (RNNs) need to adapt to new users and new gesture classes after deployment, and continual learning (CL) offers a path that avoids training from scratch. But applying CL to recurrent sEMG models is not trivial: standard recurrent training practices can undermine learning stability, and it remains an open question which CL strategy stays effective under realistic cross-user drift.

A recent study systematically evaluated three continual learning strategies under domain-incremental and class-incremental drift on the NinaPro DB1 dataset, and revealed the key impact of training patterns and feature engineering.

Side-by-Side Comparison of Continual Learning Strategies

The study compared three strategies: Experience Replay, Elastic Weight Consolidation (EWC), and Naive Fine-tuning. Under class-incremental drift, Experience Replay achieved a final accuracy of 83.3% and limited the accuracy drop caused by forgetting to 1.8%–5.8%. In contrast, EWC and naive training suffered accuracy drops as high as 16.2%–28.5%. This result indicates that in cross-user adaptation tasks, retaining and replaying old data is more important than regularizing parameters.

The advantage of Experience Replay is that it directly preserves distributional information from historical samples, mitigating catastrophic forgetting. EWC relies on parameter importance estimation, which may be insufficiently stable when differences between users are large. Naive fine-tuning leans entirely toward new data and suffers the most severe forgetting.

Experience Replay works by maintaining a memory buffer of limited capacity that stores a subset of samples from historical tasks (here, sEMG windows from old users), and mixing them with current data to form mini-batches when training on new tasks. This way, the loss function always contains gradient signals from the old distribution, constraining parameter updates from drifting away from historically optimal regions. Its effectiveness depends heavily on buffer size and sampling strategy: too small a buffer underestimates the old distribution, while too large a buffer crowds out capacity for learning new tasks.

Elastic Weight Consolidation (EWC) takes a different route that does not retain data; instead, it uses the Fisher information matrix to approximate each parameter's importance to old tasks and adds a quadratic penalty term to the loss, limiting large deviations of important parameters during new-task training. EWC's effectiveness relies on an important assumption: that parameter importance remains stable across tasks. When cross-user drift causes drastic changes in the input distribution, the Fisher estimate may become distorted, and the penalty term may instead constrain the model's ability to adapt to new users. This is one reason why EWC exhibited more severe forgetting in this study's experiments.

Catastrophic forgetting refers to the phenomenon where, when a neural network learns multiple tasks sequentially, gradient updates from new tasks overwrite the weights that old tasks depend on, causing a sharp decline in old-task performance. It is the core problem that continual learning aims to solve and the main metric for evaluating the merits of each strategy.

Training Stability: Shuffled Windows and Independent Segments

The study also found that recurrent models must be trained on segments of shuffled, independent windows. If streaming windows are fed in temporal order, the network collapses to predicting only one dominant gesture; whereas shuffled, stateless training converges stably. This finding challenges the intuition of feeding time-series data into RNNs in natural order, suggesting that in cross-user scenarios, preserving the independent and identically distributed (i.i.d.) property of samples is more critical than preserving temporal continuity.

For practitioners, this means that when building sEMG continual learning systems, one should prioritize shuffled-window data loading and avoid stateful streaming training.

The difference between stateful and stateless recurrent training lies in whether hidden states persist across batches. Stateful training passes hidden states from consecutive time steps to the next batch, which suits modeling long-range temporal dependencies but requires data to be strictly in temporal order with batch boundaries aligned to sequence boundaries. Stateless training resets hidden states to zero at the start of each batch, treating each sample window as an independent sequence.

The "network collapsing to predicting only one dominant gesture" observed in this paper is essentially the result of highly imbalanced class distributions combined with hidden-state accumulation under streaming temporal input: the model repeatedly reinforces the same output over consecutive same-class segments, gradients are dominated by the dominant class, and discriminative ability is ultimately lost. Shuffled-window stateless training makes each mini-batch approximately i.i.d., yielding lower-variance gradient estimates and more balanced classes, and thus stable convergence.

This phenomenon is closely related to backpropagation through time (BPTT), commonly used in RNN training: the longer the sequence, the more easily the chain multiplication of gradients along the time dimension leads to explosion or vanishing, whereas independent windows truncate such long-range dependencies and instead improve robustness in cross-user scenarios.

The Value and Limitations of Handcrafted Features

The study further shows that handcrafted time-domain features can improve per-task accuracy by up to about 40% compared with raw signals. However, under class-incremental drift, only Experience Replay preserves this gain. EWC and naive fine-tuning fail to maintain the advantage brought by handcrafted features, indicating that the choice of continual learning strategy directly affects the persistence of feature engineering benefits.

This result emphasizes the need for co-design of feature extraction and continual learning strategies: good features need to be paired with strategies that retain historical information, otherwise the gains will be forgotten when adapting to new users or new classes.

Handcrafted time-domain features typically include mean absolute value (MAV), waveform length (WL), zero crossings (ZC), slope sign changes (SSC), and root mean square (RMS), computed over fixed-length sliding windows to compress high-dimensional raw sEMG sequences into low-dimensional, physically meaningful descriptors. Such features are computationally cheap, highly interpretable, and somewhat robust to electrode shifts and individual differences, and have therefore long served as the default front end in resource-constrained embedded prosthetic controllers.

Compared with end-to-end learning from raw signals, handcrafted features essentially inject domain priors, reducing the representational complexity the model must discover on its own from limited data—this is the source of the roughly 40% per-task accuracy improvement in this paper. But their limitations are equally clear: once fixed, features cannot adapt to new user data; in class-incremental scenarios, if the continual learning strategy cannot preserve the feature distributions of old classes, the gains from features vanish along with forgetting. This also explains why feature engineering and CL strategies must be co-designed rather than optimized independently.

Practical Implications and Outlook

Based on the above findings, the design of cross-user sEMG gesture recognition systems should follow several principles: adopt Experience Replay as the core continual learning strategy; use shuffled-window stateless training; and incorporate handcrafted time-domain features to improve baseline performance. Future work can explore more efficient replay memory management, combine self-supervised pretraining, and validate the generalizability of these conclusions on more datasets.

With the proliferation of wearable devices and prosthetic interfaces, cross-user adaptability will be key to the practical deployment of sEMG models. This study provides a reproducible benchmark and clear strategy recommendations for landing continual learning in the physiological signal domain.

NinaPro DB1 is a widely used public benchmark dataset in the sEMG gesture recognition field, containing a series of gestures performed by multiple subjects with multi-channel electrodes on the forearm; its design naturally supports cross-subject (cross-domain) evaluation. Domain-incremental refers to new users bringing changes in input distribution while gesture classes remain unchanged, while class-incremental refers to new users simultaneously introducing new gesture classes. The two pose different challenges for continual learning: the former mainly tests stability under distribution shift, while the latter also requires the model to expand the output space without forgetting old classes, and thus carries higher forgetting risk. This paper evaluates under both types of drift precisely to distinguish the applicable boundaries of strategies in different real-world scenarios.

One-line Summary

A systematic evaluation of continual learning strategies finds Experience Replay plus shuffled-window training best suits cross-user sEMG recognition.

Paragraph Summary

This paper systematically evaluates three continual learning strategies—Experience Replay, Elastic Weight Consolidation (EWC), and Naive Fine-tuning—for cross-user sEMG gesture recognition. Based on the NinaPro DB1 dataset, the study finds that Experience Replay achieves 83.3% final accuracy under class-incremental drift with only 1.8%–5.8% forgetting, significantly outperforming EWC and naive fine-tuning at 16.2%–28.5%. The study also reveals that recurrent models must use shuffled-window stateless training, as streaming temporal input causes network collapse; handcrafted time-domain features can improve per-task accuracy by about 40%, but only Experience Replay maintains this gain. The conclusions indicate that retaining historical data is more effective than parameter regularization in cross-user adaptation, and provide clear strategy recommendations for system design.