PPG-Bench Revealed: Are Mainstream PPG Foundation Models Truly General-Purpose?

When Photoplethysmography Meets Foundation Models

Photoplethysmography (PPG) signals are being captured at a remarkable scale by wearable devices and clinical instruments alike. From smartwatches to hospital monitors, these devices continuously record optical signals reflecting blood vessel volume changes with each heartbeat. As the volume of signals has exploded, researchers have developed an increasing number of PPG Foundation Models (FMs), aiming to learn reusable signal representations from massive datasets to serve a wide range of downstream tasks.

The vision is appealing: train one powerful foundation model that generalizes across tasks like cardiovascular state estimation, sleep staging, and identity recognition. But validating that vision in practice is difficult — different studies use different datasets, task definitions, preprocessing pipelines, downstream model heads, hyperparameter tuning strategies, and baselines, making cross-study comparison nearly impossible. PPG-Bench was created precisely to address this fragmented landscape.

The underlying principle of PPG signal acquisition is straightforward: an LED emits light of a specific wavelength into skin tissue, and a photodetector captures changes in light intensity after absorption or reflection by the tissue. Because hemoglobin's light absorption varies periodically with blood vessel volume changes driven by the heartbeat, the photodetector's output maps to the pulse rhythm. This principle enables consumer-grade devices like smartwatches and pulse oximeters to continuously monitor heart rate, blood oxygen saturation, and other physiological metrics without any blood contact. PPG waveforms contain rich physiological information: peak-to-peak intervals reflect heart rate, waveform morphology correlates with vascular elasticity and blood pressure, and respiratory motion leaves traces in baseline drift. This information density is precisely what motivates researchers to use foundation models for unified multi-dimensional representation extraction, rather than designing separate feature engineering pipelines for each task.

What PPG-Bench Does

PPG-Bench is an open-source unified evaluation benchmark with impressive scope. The research team selected 5 publicly available PPG foundation models and systematically evaluated them across 9 openly accessible datasets on 26 tasks. These tasks span multiple application domains: cardiovascular state estimation, activity recognition, sleep monitoring, disease screening, and identity recognition.

To comprehensively characterize model adaptability, the benchmark defines four progressively flexible downstream training paradigms:

• Linear Probing: The simplest approach — trains only a linear classifier on frozen features
• Random Forests: Non-linear but leaves the representation unchanged
• Frozen MLP Probes: Increases downstream head capacity without modifying the backbone
• End-to-end Finetuning: The most flexible — allows all parameters to be updated

Each paradigm is paired with model-specific hyperparameter optimization and subject-disjoint cross-validation, ensuring rigor and reproducibility. This design prevents information leakage that would occur if the same subject's data appeared in both training and test sets.

Subject-disjoint splitting is a critical principle in health signal evaluation. Physiological signals have strong individual specificity — if the same subject appears in both training and test sets, the model learns to "recognize the person" rather than "understand physiological patterns," inflating test metrics and undermining generalization. In wearable health research, this data leakage issue is especially pronounced, since inter-individual differences (e.g., baseline heart rate, waveform morphology) often far exceed task-relevant differences (e.g., disease status). Strict subject-disjoint splits ensure the model must predict on individuals it has never seen — which is the appropriate standard for measuring true representation transferability.

Three Baseline Comparisons: A Revealing Design

What gives this work real weight is its three carefully chosen baseline comparisons. A common flaw in foundation model evaluations is comparing only against other deep models, without asking whether simpler methods perform just as well. PPG-Bench introduces three clearly differentiated reference points:

1. Random/Constant Baseline: Contains no useful information — the lowest possible reference point, used to verify whether a model actually learned anything
2. Deterministic PRV Feature Set: Traditional handcrafted features based on Pulse-Rate-Variability, requiring no learning whatsoever
3. General-purpose time-series foundation model MOMENT: Not designed specifically for PPG, but represents general time-series modeling capability

These three baselines form a clear evaluation logic: a qualified PPG-specific foundation model should at minimum outperform the random baseline, and more importantly should beat both handcrafted features and the general-purpose time-series model — otherwise, its very existence is questionable.

Pulse-Rate-Variability (PRV) is the PPG-domain analogue of Heart-Rate-Variability (HRV), referring to the beat-to-beat variation in time intervals between adjacent pulse wave peaks (PP intervals). Traditional HRV analysis originated from RR intervals in ECG signals, but because PPG is far easier to acquire via wearables, PRV is widely used as a practical substitute for HRV. PRV features typically include time-domain metrics (e.g., SDNN, RMSSD), frequency-domain metrics (low-frequency/high-frequency power ratios), and nonlinear metrics (e.g., approximate entropy, Poincaré plot parameters) — all of which can be deterministically computed from the signal without any training. Using a PRV feature set as a baseline is a direct test of the fundamental question: do the deep representations from foundation models actually outperform human prior knowledge?

Sobering Findings

The evaluation results are striking. Across all 68 task-paradigm combinations:

• In 35.3% of combinations, PPG foundation models showed no meaningful advantage over the random/constant baseline — meaning in more than one-third of cases, these models performed no better than random guessing;
• In 83.8% of combinations, no PPG-specific foundation model was able to simultaneously outperform both PRV handcrafted features and the general model MOMENT.

This means that in the vast majority of tasks, PPG-specific models trained at great computational cost are no stronger than simple pulse-rate-variability features or a non-specialized general-purpose time-series model. The study also surfaces an intriguing finding: increasing downstream head capacity generally improves performance, but when the foundation model itself operates at random/constant levels, adding capacity actually leads to significant performance degradation. This suggests that many apparent performance gains may be attributed to the downstream head rather than the representation quality of the foundation model itself.

Implications for the Field

The research team's conclusion is direct: current PPG foundation models cannot yet claim to provide general-purpose, transferable signal representations. This is not a rejection of the foundation model approach itself, but a serious reflection on current evaluation practices.

The work's most important methodological contribution is its emphasis on the indispensability of simple comparison baselines. As deep learning models grow ever more complex, researchers frequently neglect comparisons with non-learning methods. Even if a model achieves impressive absolute numbers, if it cannot beat handcrafted features, its claim to have "learned useful representations" is not credible. PPG-Bench calls on future evaluations in this field to consistently include simple, non-learning baselines as reference checks.

For practitioners, this is also a reminder to stay clear-headed when evaluating new models: when a new PPG foundation model is released, it's worth asking first — has it been compared against PRV features? The benchmark code and instructions for evaluating additional models are open-sourced at https://github.com/mannas-ai/ppg-bench, and interested researchers can incorporate their own models into this unified evaluation framework.

Summary

PPG-Bench provides a unified, rigorous, and open-source yardstick for the fragmented landscape of PPG foundation model evaluation. Through solid experiments across 68 task-paradigm combinations, it demonstrates a somewhat uncomfortable truth: today's PPG-specific foundation models are still far from fulfilling their promise of "general-purpose transferability." This kind of self-reflective benchmark work is precisely the force needed to drive healthy progress in the field.