CapMem
A benchmark dataset for evaluating visual-language models on episodic memory understanding in long videos. It contains 75 videos, 1,000 multiple-choice questions, covers 16 scene types, totals 33.7 hours, and centers on episodic memory video captioning question answering.
Timeline (last 90 days)
CapMem proposes converting long video content into text captions as a reusable episodic memory medium for long-video episodic reasoning.
CapMem defines a new task: Episodic Memory Video Caption QA.
The CapMem benchmark dataset contains 75 videos with a total duration of 33.7 hours, 1,000 manually annotated multiple-choice questions, covering 16 different scenarios.
Under the 30-second caption window, 10 out of 12 models performed better; under the 60-second caption window, 8 out of 12 models performed better.
In matched-frame controlled experiments conducted on six Qwen-series models, the caption approach improved average accuracy by 3.22 points and 2.55 points respectively under frame-aligned conditions.
The caption-guided retrieve-and-verify framework designed by the research team further improved accuracy by up to 5.3 points.
The caption approach has limitations: caption generation quality and visual details that cannot be precisely described in language may become information bottlenecks.
All Facts (7)
CapMem proposes converting long video content into text captions as a reusable episodic memory medium for long-video episodic reasoning.
50%UnverifiedCapMem defines a new task: Episodic Memory Video Caption QA.
50%UnverifiedThe CapMem benchmark dataset contains 75 videos with a total duration of 33.7 hours, 1,000 manually annotated multiple-choice questions, covering 16 different scenarios.
50%UnverifiedUnder the 30-second caption window, 10 out of 12 models performed better; under the 60-second caption window, 8 out of 12 models performed better.
50%UnverifiedIn matched-frame controlled experiments conducted on six Qwen-series models, the caption approach improved average accuracy by 3.22 points and 2.55 points respectively under frame-aligned conditions.
50%UnverifiedThe caption-guided retrieve-and-verify framework designed by the research team further improved accuracy by up to 5.3 points.
50%UnverifiedThe caption approach has limitations: caption generation quality and visual details that cannot be precisely described in language may become information bottlenecks.
50%