EmoHarness
An external, plug-in framework for improving the multimodal emotion understanding of small models. It organizes evidence (transcripts, prosodic descriptions, visual observations) and guides the model to compare evidence, improving recognition accuracy without updating model weights.
Timeline (last 90 days)
EmoHarness is an external framework that guides a frozen small model to use multimodal evidence by organizing prompts, settings, and verification code, without updating model weights.
EmoHarness's initial scaffold numbers the student model's visual observations, transcript text, and prosody descriptions as citable evidence, and requires the student model to compare candidate emotions against the evidence one by one.
On 5,507 MME-Emotion samples, EmoHarness improved the task macro-average recognition rate of four student models and two Builders by 7.6 to 13.4 percentage points compared with single calls using video only.
Transcript text and prosody descriptions contributed most of EmoHarness's gains, with improvements of +5.38 to +10.71 percentage points.
The scaffold's effect on audio-enhanced calls ranged from -0.79 to +2.54 percentage points.
The emotional context provided by the EmoHarness framework transferred to the IntentQA task, with three of the four student models showing small improvements, but these did not pass corrected tests.
EmoHarness's gains come mainly from providing and organizing audio-derived evidence (transcripts and prosody descriptions), rather than from the framework's automatic evolution.
The actual effect of EmoHarness's evolution mechanism did not reach statistical significance, and the framework's transfer ability was verified only on some models.
All Facts (8)
EmoHarness is an external framework that guides a frozen small model to use multimodal evidence by organizing prompts, settings, and verification code, without updating model weights.
50%UnverifiedEmoHarness's initial scaffold numbers the student model's visual observations, transcript text, and prosody descriptions as citable evidence, and requires the student model to compare candidate emotions against the evidence one by one.
50%UnverifiedOn 5,507 MME-Emotion samples, EmoHarness improved the task macro-average recognition rate of four student models and two Builders by 7.6 to 13.4 percentage points compared with single calls using video only.
50%UnverifiedTranscript text and prosody descriptions contributed most of EmoHarness's gains, with improvements of +5.38 to +10.71 percentage points.
50%UnverifiedThe scaffold's effect on audio-enhanced calls ranged from -0.79 to +2.54 percentage points.
50%UnverifiedThe emotional context provided by the EmoHarness framework transferred to the IntentQA task, with three of the four student models showing small improvements, but these did not pass corrected tests.
50%UnverifiedEmoHarness's gains come mainly from providing and organizing audio-derived evidence (transcripts and prosody descriptions), rather than from the framework's automatic evolution.
50%UnverifiedThe actual effect of EmoHarness's evolution mechanism did not reach statistical significance, and the framework's transfer ability was verified only on some models.
50%