PPG-Bench
An open-source unified evaluation benchmark for photoplethysmography (PPG) foundation models. It covers multiple PPG foundation models, datasets, and downstream tasks, and supports systematic assessment across various training paradigms.
Timeline (last 90 days)
PPG-Bench is an open-source unified evaluation benchmark designed to systematically assess PPG foundation models.
PPG-Bench selects 5 publicly available PPG foundation models and evaluates them across 26 tasks on 9 openly accessible datasets.
PPG-Bench's evaluation designs four progressively complex downstream training paradigms: linear probing, random forest, frozen MLP probe, and end-to-end fine-tuning.
PPG-Bench adopts subject-disjoint cross-validation to prevent information leakage.
PPG-Bench introduces three control baselines: a random/constant baseline, a deterministic handcrafted feature set based on pulse rate variability (PRV), and the general time-series foundation model MOMENT.
Across all 68 task-paradigm combinations, in 35.3% of them PPG foundation models show no meaningful advantage over the random/constant baseline.
In 83.8% of task-paradigm combinations, no PPG-specific foundation model is able to simultaneously outperform both the PRV handcrafted features and the general model MOMENT.
Increasing the capacity of the downstream head generally improves performance, but if the model itself operates only at the random/constant level, increasing capacity instead leads to significant performance degradation.
The research team concludes that current PPG foundation models cannot yet claim to provide general and transferable signal representations.
The benchmark code and evaluation instructions for PPG-Bench have been open-sourced at https://github.com/mannas-ai/ppg-bench.
1 more timeline events
All Facts (11)
Foundation model evaluations should include simple non-learning control checks (such as handcrafted features); if a model cannot even outperform handcrafted features, its claim to have learned useful representations is untenable.
50%UnverifiedPPG-Bench selects 5 publicly available PPG foundation models and evaluates them across 26 tasks on 9 openly accessible datasets.
50%UnverifiedPPG-Bench's evaluation designs four progressively complex downstream training paradigms: linear probing, random forest, frozen MLP probe, and end-to-end fine-tuning.
50%UnverifiedPPG-Bench adopts subject-disjoint cross-validation to prevent information leakage.
50%UnverifiedPPG-Bench introduces three control baselines: a random/constant baseline, a deterministic handcrafted feature set based on pulse rate variability (PRV), and the general time-series foundation model MOMENT.
50%UnverifiedAcross all 68 task-paradigm combinations, in 35.3% of them PPG foundation models show no meaningful advantage over the random/constant baseline.
50%UnverifiedIn 83.8% of task-paradigm combinations, no PPG-specific foundation model is able to simultaneously outperform both the PRV handcrafted features and the general model MOMENT.
50%UnverifiedIncreasing the capacity of the downstream head generally improves performance, but if the model itself operates only at the random/constant level, increasing capacity instead leads to significant performance degradation.
50%UnverifiedThe research team concludes that current PPG foundation models cannot yet claim to provide general and transferable signal representations.
50%UnverifiedPPG-Bench is an open-source unified evaluation benchmark designed to systematically assess PPG foundation models.
50%UnverifiedThe benchmark code and evaluation instructions for PPG-Bench have been open-sourced at https://github.com/mannas-ai/ppg-bench.
50%