[KongchangAI]
ConceptMultimodal Emotion Recognition

Multimodal Emotion Recognition

A core task in affective computing that integrates text, speech, visual and other modality information to determine a speaker's emotional state, and that faces the challenge of inter-modality conflict.

Timeline (last 90 days)

Nov 10

EmoHarness is an external framework that guides a frozen small model to use multimodal evidence by organizing prompts, settings, and verification code, without updating model weights.

Unverified50%
Nov 10

In multimodal emotion understanding, when textual, vocal, and visual cues conflict, small models with limited parameters are prone to misjudgment driven by a single modality.

Unverified50%
Nov 10

Under all three source mappings, replacing the original audio-video with unrelated audio-video (RandA/V) reduced accuracy relative to the full input in all 18 comparisons.

Unverified50%
Nov 10

The accuracy drops caused by unrelated audio-video replacement in all 18 comparisons remained statistically significant under conversation-cluster inference and BH-FDR correction.

Unverified50%
Nov 10

In all six model-dataset pairings, adding the target audio (text+audio) improved emotion recognition accuracy.

Unverified50%
Nov 10

For deployers, one cannot assume that all models utilize audio to the same degree; it is worth verifying item by item when selecting models.

Unverified50%
Nov 10

The "presence or absence" of multimodal capability is no longer a distinguishing point; whether a model can remain stable under misleading inputs is the focus of the next stage of evaluation.

Unverified50%
Nov 10

The performance drop caused by unrelated audio-video replacement (mismatch-specific harm) differs from simple modality absence: the model fuses mismatched audio-video as valid evidence.

Unverified50%
Nov 10

Of the six model-dataset pairings, only four retained a significant audio gain after FDR correction.

Unverified50%

All Facts (9)

Unverified

In multimodal emotion understanding, when textual, vocal, and visual cues conflict, small models with limited parameters are prone to misjudgment driven by a single modality.

50%
Unverified

EmoHarness is an external framework that guides a frozen small model to use multimodal evidence by organizing prompts, settings, and verification code, without updating model weights.

50%
Unverified

Under all three source mappings, replacing the original audio-video with unrelated audio-video (RandA/V) reduced accuracy relative to the full input in all 18 comparisons.

50%
Unverified

The accuracy drops caused by unrelated audio-video replacement in all 18 comparisons remained statistically significant under conversation-cluster inference and BH-FDR correction.

50%
Unverified

In all six model-dataset pairings, adding the target audio (text+audio) improved emotion recognition accuracy.

50%
Unverified

For deployers, one cannot assume that all models utilize audio to the same degree; it is worth verifying item by item when selecting models.

50%
Unverified

The performance drop caused by unrelated audio-video replacement (mismatch-specific harm) differs from simple modality absence: the model fuses mismatched audio-video as valid evidence.

50%
Unverified

Of the six model-dataset pairings, only four retained a significant audio gain after FDR correction.

50%
Unverified

The "presence or absence" of multimodal capability is no longer a distinguishing point; whether a model can remain stable under misleading inputs is the focus of the next stage of evaluation.

50%

Source Articles