[KongchangAI]
· 2 min read· 1,168 words

Emotional Inertia-Driven Framework: A New Approach to Multimodal Depression Detection, InertiaMPDD

Emotional Inertia-Driven Framework: A New Approach to Multimodal Depression Detection, InertiaMPDD

InertiaMPDD uses a world-model-inspired latent predictor to learn healthy emotional dynamics, replacing direct label prediction with deviation evidence.

InertiaMPDD addresses multimodal depression screening's lack of deviation evidence by using an emotional inertia-driven world-model-inspired latent predictor: it first learns how emotions continue in a healthy state, then judges individual deviation. It constructs reference deviation, residual decay rate, and Jacobian spectrum descriptors as candidate surrogate biomarkers. Fusion achieves 98.41% F1 on D-Vlog and 84.98% on LMVD across five cohorts, but descriptor utility is scenario-dependent.

Multimodal depression screening typically estimates risk from speech, visual behavior, and language, but most existing methods only perform label prediction and lack evidence characterizing "deviation from a healthy state"; at the same time, recording-level supervision leaves fine-grained emotional dynamics unlabeled. Motivated by emotional inertia (the persistence of emotions), InertiaMPDD uses a world-model-inspired latent predictor to learn the next-state patterns of healthy states, offering a new perspective for personalized depression detection.

InertiaMPDD multimodal depression detection framework diagram

From "Predicting Labels" to "Characterizing Deviation": A Shift in Problem Perspective

The current mainstream approach to multimodal depression screening is to concatenate speech, facial visual behavior, and language text into features and then predict a label. This approach can provide a risk judgment, but it struggles to answer another question: how far is this person's emotional dynamics from a "non-depressed reference state"? The paper points out that recording-level supervision provides only an overall label, with no annotations for fine-grained emotional changes, making it difficult for models to learn interpretable evidence of dynamic deviation. InertiaMPDD's starting point is precisely to turn "deviation from a healthy reference" from an implicit assumption into a modelable, measurable object.

World-Model-Inspired Latent Predictor: Learning the "Healthy Next State"

At the core of InertiaMPDD is a world-model-inspired latent predictor. It does not directly fit depression labels; instead, it uses pretrained multimodal representations to predict the next state from observed history and context, with healthy state patterns as the learning target. This design transforms emotional inertia—the persistence of emotions over time—into learnable dynamic regularities: the model first learns "how normal emotions continue," and then judges whether an individual's dynamics are anomalous based on this. This path of first building a reference and then examining deviation is clearly distinct from the paradigm of direct classification.

The paper thereby constructs three types of reference-derived emotional dynamics descriptors: reference deviation, residual decay rate, and a descriptor based on the Jacobian spectrum. These are evaluated as candidate surrogate biomarkers, usable both for standalone screening and for personalized decision correction. In other words, these descriptors are not merely features; they attempt to carry the semantics of "the degree and manner of deviation from a healthy reference."

A world model is a modeling paradigm widely studied in reinforcement learning and sequential decision-making. Its core idea is to let an agent learn the dynamic evolution of the environment in latent space—given the current state and action, predict the next state—so as to perform planning or anomaly detection without relying on real interaction. Transferring this idea to depression screening means the model no longer directly fits labels; instead, it first learns the dynamic regularities of "how healthy emotional states evolve over time." When an individual's actual emotional trajectory systematically deviates from this prediction, the deviation itself becomes evidence of depression risk. This paradigm of "first building a reference, then examining deviation" stands in sharp contrast to end-to-end classifiers that delegate all discriminative pressure to label supervision, and it also gives the model's intermediate quantities interpretable clinical meaning.

The Jacobian spectrum descriptor originates from the analysis of local stability in latent dynamical systems. Specifically, the model computes the Jacobian matrix of the state transition function, and its eigenvalues (spectrum) characterize the contraction or divergence tendency of the system near the reference point: an eigenvalue magnitude less than 1 means perturbations decay over time and emotions return to baseline, while greater than 1 means deviations are amplified and emotional dynamics may become unstable. In InertiaMPDD, the Jacobian spectrum descriptor is precisely a stability indicator used to quantify whether this "emotional inertia is abnormal," complementing reference deviation (magnitude of deviation) and residual decay rate (speed of return), together forming a multidimensional characterization of emotional dynamic deviation. This also explains why the three descriptors need to be evaluated jointly—they correspond respectively to the size of deviation, the speed of recovery, and the stability of the system.

Multi-Cohort Experimental Results: Fusion Is Effective, but Descriptor Utility Is Scenario-Dependent

The paper evaluates on five cohorts from four datasets. Multimodal fusion achieves 98.41% F1 on D-Vlog and 84.98% on LMVD. These two numbers show that the fusion path has strong discriminative ability under specific data distributions.

However, the descriptors do not perform consistently. On MPDD, standalone descriptors outperform fixed additive fusion; on AVEC2014, the descriptors have weak discriminative ability. The paper thereby proposes a key distinction: reference informativeness and fusion benefit are not the same thing, and the utility of descriptors is context-dependent. This finding is important for practical deployment—one cannot assume that "fusion is always better," nor that dynamic descriptors are reliable across all cohorts.

The five cohorts in the paper come from four datasets: D-Vlog, LMVD, MPDD, and AVEC2014. These datasets differ significantly in modality composition, recording conditions, and label definitions. AVEC2014 is an earlier audio-visual depression challenge dataset with a smaller sample size and relatively constrained recording conditions; D-Vlog and LMVD come from video blog scenarios, with data distributions different from the former. Therefore, the weaker discriminative performance of descriptors on AVEC2014 may reflect both insufficient reference informativeness and the impact of cross-cohort distribution shift. This suggests that when evaluating dynamic descriptors, the reference quality and scenario characteristics of the dataset itself must be taken into account, rather than judging method quality solely by F1 on a single cohort.

Implications and Limitations for Personalized Detection

The value of InertiaMPDD lies in advancing depression detection from single-point label prediction to modeling and measuring emotional dynamic deviation. Reference deviation, residual decay rate, and Jacobian spectrum descriptors provide interpretable intermediate quantities, and personalized decision correction allows these quantities to participate in the final judgment. At the same time, the paper honestly presents limitations: descriptors have weak discriminative ability on AVEC2014, indicating that their generalization depends on data and scenario; fine-grained dynamics remain unlabeled under recording-level supervision, and more validation is needed between the dynamic patterns learned by the model and clinically meaningful emotional processes.

For multimodal mental health computing, this work suggests a promising line of research: first establish a healthy reference, then use dynamic deviation as evidence, rather than delegating all pressure to end-to-end classifiers.

One-line Summary

InertiaMPDD uses a world-model-inspired latent predictor to learn healthy emotional dynamics, replacing direct label prediction with deviation evidence.

Paragraph Summary

Addressing the problem in multimodal depression screening of "predicting labels only, lacking deviation evidence," InertiaMPDD proposes a world-model-inspired latent predictor motivated by emotional inertia: the model does not directly fit depression labels but first learns how emotions continue in a healthy state, then judges whether an individual's dynamics deviate. The paper thereby constructs three types of descriptors—reference deviation, residual decay rate, and Jacobian spectrum—as candidate surrogate biomarkers for standalone screening and personalized decision correction. On five cohorts from four datasets, multimodal fusion achieves 98.41% F1 on D-Vlog and 84.98% on LMVD, but descriptor utility is scenario-dependent: on MPDD, standalone descriptors outperform fixed additive fusion, while on AVEC2014 they are weak. The study thereby distinguishes reference informativeness from fusion benefit, suggesting that one cannot assume fusion or dynamic descriptors are universally superior.

#multimodal depression detection#emotional inertia#world model#InertiaMPDD#Jacobian spectrum#depression screening#latent predictor
Share:

Related articles