[KongchangAI]
· 1 min read· 805 words

Bio-PM Encoder Space Synthetic IMU Data Significantly Improves Cross-Subject HAR Accuracy

Bio-PM Encoder Space Synthetic IMU Data Significantly Improves Cross-Subject HAR Accuracy

Synthesizing IMU data in Bio-PM's frozen encoder token space raises LOSO macro-F1 from 0.729 to 0.809.

This article introduces a new study addressing IMU data scarcity and cross-subject generalization in HAR by generating token sequences in the frozen representation space of the pretrained biosignal encoder Bio-PM rather than at the raw signal level. The authors train autoregressive Transformer and diffusion class-conditional decoders, with PatchDecoder for approximate waveform reconstruction. Evaluation includes distribution metrics, embedding visualization, and LOSO classification. Results show the diffusion decoder achieves TSTR macro-F1 of 0.68; combining synthetic token augmentation with 320-dim gravity-aware features raises WISDM 51-subject LOSO macro-F1 from 0.729 to 0.809, with gains concentrated in the most challenging activity classes and statistically significant. The authors note the path is low-cost and feasible but still depends on a specific encoder and dataset, waveform reconstruction is only approximate, and encoder generalization and computational overhead must be assessed before deployment.

Why Synthetic IMU Data Is Worth Attention

Human activity recognition (HAR) relies on inertial measurement unit (IMU) data, but collecting and annotating this data is costly, and models that perform well on training subjects often fail when applied to new people—cross-subject generalization is a long-standing pain point. Even with balanced data classes, classifiers still struggle to distinguish dynamically similar activities. A new study proposes generating token sequences not at the raw signal level, but within the representation space of the pretrained biosignal encoder Bio-PM, as a practical source of data augmentation.

Bio-PM-guided synthetic IMU generation pipeline

Cross-subject generalization is a core evaluation paradigm in HAR: models are trained on data from some subjects and tested on unseen subjects (i.e., leave-one-subject-out, LOSO). Because each person's gait, limb length, and wearing habits differ, IMU signal distributions exhibit significant individual variation, and models easily overfit to patterns specific to training subjects. LOSO macro-F1 is a common metric for measuring this capability—macro-F1 computes F1 separately for each activity class and then averages them, making it more sensitive to minority and hard-to-classify classes, and thus able to reflect whether augmentation truly helps difficult samples rather than being masked by majority classes.

Method: Training Decoders in a Frozen Encoder Space

The research team used Bio-PM's frozen encoder to convert 10-second IMU windows into token embedding sequences, then trained two types of class-conditional decoders in this space:

  • Autoregressive Transformer decoder: predicts tokens sequentially;
  • Diffusion decoder: learns the token distribution through a denoising objective.

In addition, the authors proposed PatchDecoder for approximate reconstruction of raw waveforms. Evaluation dimensions included distribution metrics and embedding space visualization (to judge generation realism), as well as leave-one-subject-out (LOSO) HAR classification (to judge downstream utility).

According to the paper, the diffusion decoder achieved a macro-F1 of 0.68 under the "train on synthetic, test on real" (TSTR) setting, indicating that the generated tokens are realistic enough to train a classifier that transfers to real data.

Bio-PM (biosignal pretrained model) is a class of encoders self-supervised pretrained on large-scale physiological/biosignal data, capable of mapping raw time-series signals into token embedding sequences with semantic structure. The phrase "frozen encoder" in this article means that encoder parameters are not updated during the generation stage; the decoder is trained only on a fixed representation space, which both reduces training cost and ensures that generated samples and real samples lie in the same feature distribution.

The autoregressive Transformer decoder predicts the next token one by one in sequence order, similar to GPT-style generation; the diffusion decoder starts from noise and approximates the target token distribution through multi-step denoising. Both are class-conditional generation, i.e., generating corresponding tokens given an activity class label, thereby enabling on-demand synthesis of balanced data or supplementation of difficult classes.

TSTR (Train on Synthetic, Test on Real) is a standard setting for evaluating the utility of generated data: a classifier is trained solely on synthetic data and then tested on real data. The higher the TSTR macro-F1, the closer the synthetic tokens are to the real distribution and the more transferable discriminative information they contain.

Key Results: LOSO Macro-F1 Rises from 0.729 to 0.809 After Augmentation

On the 51 subjects of the WISDM benchmark, combining synthetic token augmentation with a 320-dimensional gravity-aware feature representation raised LOSO macro-F1 from 0.729 with real data only to 0.809, with statistically significant gains concentrated in the most challenging activity classes. This means that representation-space synthesis can indeed alleviate the cross-subject generalization problem, especially for hard-to-classify samples.

The 320-dimensional gravity-aware feature representation is a handcrafted IMU feature engineering method that typically decomposes acceleration signals into gravity and linear acceleration components, extracts statistics (mean, variance, frequency-domain energy, etc.) over multiple time windows, and finally concatenates them into a fixed-dimensional feature vector. Compared with end-to-end deep models, such features are more robust to individual differences and are often used as strong HAR baselines. This article combines them with synthetic token augmentation, showing that representation-space synthetic samples can complement traditional feature representations and bring additional gains.

WISDM is a publicly available dataset widely used in the HAR field, containing phone/watch IMU recordings from multiple subjects during activities such as walking, running, going upstairs, going downstairs, sitting, and standing. It has a relatively large number of subjects, making it suitable for LOSO evaluation; the macro-F1 improvement is statistically significant across 51 subjects and concentrated in hard-to-classify classes, indicating that the augmentation is not simply amplifying easy-to-classify samples.

Implications and Limitations

This work validates the feasibility of "generating in the representation space of a high-fidelity pretrained encoder," providing a low-cost augmentation path for scenarios with scarce IMU data. However, current results still depend on a specific encoder and dataset, and the reconstruction from generated tokens to raw waveforms is only approximate; practical deployment requires evaluating the encoder's generalization ability and computational overhead.

Paper source screenshot

#IMU data augmentation#cross-subject HAR#Bio-PM encoder#LOSO macro-F1#diffusion decoder#synthetic sensor data
Share:

Related articles