ScaleBrain
A scale-consistent pretraining framework for electroencephalography (EEG) foundation models. It employs multi-scale EEG tokenization and scale-specific reconstruction to co-design inputs and targets, and uses axial spatio-temporal attention to fuse multi-scale representations.
Timeline (last 90 days)
The researchers propose ScaleBrain, a scale-consistent pretraining framework that explicitly coordinates multi-scale representations from both the model input and the optimization objective.
ScaleBrain introduces a multi-scale EEG tokenization method that represents the same EEG signal through analysis windows of different time spans and shared temporal anchors, so that the model input naturally possesses scale structure.
ScaleBrain adopts scale-specific reconstruction as the self-supervised objective to ensure that each temporal resolution receives sufficient supervisory signal.
ScaleBrain designs an axial spatio-temporal attention mechanism that performs context modeling on the fused multi-scale representations along both the spatial and temporal dimensions, capturing inter-electrode spatial correlations and temporal dynamic dependencies respectively.
ScaleBrain was extensively evaluated on six public BCI benchmark datasets and compared with 12 representative baseline methods.
ScaleBrain achieved the highest mean Balanced Accuracy on five datasets and ranked third on the remaining one dataset.
ScaleBrain contains only 5.0M parameters.
Axial attention has mature applications in video understanding and high-dimensional image generation, and ScaleBrain transfers it to EEG multi-scale representation fusion.
All Facts (8)
The researchers propose ScaleBrain, a scale-consistent pretraining framework that explicitly coordinates multi-scale representations from both the model input and the optimization objective.
50%UnverifiedScaleBrain introduces a multi-scale EEG tokenization method that represents the same EEG signal through analysis windows of different time spans and shared temporal anchors, so that the model input naturally possesses scale structure.
50%UnverifiedScaleBrain adopts scale-specific reconstruction as the self-supervised objective to ensure that each temporal resolution receives sufficient supervisory signal.
50%UnverifiedScaleBrain designs an axial spatio-temporal attention mechanism that performs context modeling on the fused multi-scale representations along both the spatial and temporal dimensions, capturing inter-electrode spatial correlations and temporal dynamic dependencies respectively.
50%UnverifiedScaleBrain was extensively evaluated on six public BCI benchmark datasets and compared with 12 representative baseline methods.
50%UnverifiedScaleBrain achieved the highest mean Balanced Accuracy on five datasets and ranked third on the remaining one dataset.
50%UnverifiedScaleBrain contains only 5.0M parameters.
50%UnverifiedAxial attention has mature applications in video understanding and high-dimensional image generation, and ScaleBrain transfers it to EEG multi-scale representation fusion.
50%