OmniGround
A general-purpose omni-modal large model with omni-modal temporal grounding capability that, through attention consistency training (TACG), outputs event start and end timestamps on a unified timeline of video, audio and text.
Timeline (last 90 days)
The OmniGround team built a 580K-sample training dataset by reconstructing temporal question answering from dense audio-visual annotations and organizing general omni-modal instruction data.
OmniGround is a general omni-modal model trained on TACG and the 580K dataset, with stronger temporal grounding capability.
OmniGround reaches 37.5 mIoU on the LongVALE benchmark and 35.7 mIoU on the UnAV-100 benchmark, both achieving SOTA levels for omni-modal temporal grounding.
OmniGround remains competitive on general audio-visual understanding and vision-only grounding tasks, without sacrificing other capabilities for specializing in temporal grounding.
All Facts (4)
The OmniGround team built a 580K-sample training dataset by reconstructing temporal question answering from dense audio-visual annotations and organizing general omni-modal instruction data.
50%UnverifiedOmniGround is a general omni-modal model trained on TACG and the 580K dataset, with stronger temporal grounding capability.
50%UnverifiedOmniGround reaches 37.5 mIoU on the LongVALE benchmark and 35.7 mIoU on the UnAV-100 benchmark, both achieving SOTA levels for omni-modal temporal grounding.
50%UnverifiedOmniGround remains competitive on general audio-visual understanding and vision-only grounding tasks, without sacrificing other capabilities for specializing in temporal grounding.
50%