Vid-LLM
A video large language model capable of jointly understanding video content and natural language queries. It supports multiple video-language tasks, including video question answering and video description generation, functioning as a multimodal large language model for video-language alignment.
Timeline (last 90 days)
Discriminative methods that have Vid-LLMs directly judge frame relevance suffer from an Interpretation–Selection Gap, where the model understands the query but may not accurately execute the selection action.
RACER decomposes the frame selection task into two steps: a lightweight Vid-LLM is responsible for rewriting complex queries into a series of sub-queries, and an embedding model serves as a retrieval tool to locate relevant evidence frames.
Even when using components with limited capability, RACER can still achieve effective frame selection, and weaker components can synergistically enhance more powerful Vid-LLMs.
All Facts (3)
Discriminative methods that have Vid-LLMs directly judge frame relevance suffer from an Interpretation–Selection Gap, where the model understands the query but may not accurately execute the selection action.
50%UnverifiedRACER decomposes the frame selection task into two steps: a lightweight Vid-LLM is responsible for rewriting complex queries into a series of sub-queries, and an embedding model serves as a retrieval tool to locate relevant evidence frames.
50%UnverifiedEven when using components with limited capability, RACER can still achieve effective frame selection, and weaker components can synergistically enhance more powerful Vid-LLMs.
50%