ConceptAudio-Visual Captioning
Audio-Visual Captioning
A task that requires a model to watch a video while listening to its audio and then generate natural language captions describing the video content. It involves sub-abilities such as visual object recognition, audio event detection, temporal grounding and cross-modal association, and serves as a probe task for diagnosing the overall perceptual capability of multimodal models.
Timeline (last 90 days)
Oct 12
OmniCapBench (Omni-Video Caption Benchmark) is a deeply structured diagnostic framework for audio-visual caption evaluation, whose prediction targets are three groups of atomic, verifiable evaluation units: entity reference, visual shot, and audio event.
Unverified50%