MLLM-as-a-Judge
The use of a multimodal large language model as a judge to perform semantic-level assessment of retrieval results rather than relying solely on annotated labels. This allows genuine errors to be distinguished from plausible matches missed by annotation, and is used to diagnose the root causes of embedding model failures.
Timeline (last 90 days)
The researchers introduced an MLLM-as-a-Judge diagnostic framework that breaks down top-1 retrieval behavior into three error sources: labeled positives, false negatives, and hard/easy negative errors.
MLLM-as-a-Judge refers to using a multimodal large model as the judge to make semantic-level determinations of retrieval results rather than relying only on annotation labels, thereby distinguishing 'genuine errors' from 'reasonable matches missed by annotation'.
All Facts (2)
The researchers introduced an MLLM-as-a-Judge diagnostic framework that breaks down top-1 retrieval behavior into three error sources: labeled positives, false negatives, and hard/easy negative errors.
50%UnverifiedMLLM-as-a-Judge refers to using a multimodal large model as the judge to make semantic-level determinations of retrieval results rather than relying only on annotation labels, thereby distinguishing 'genuine errors' from 'reasonable matches missed by annotation'.
50%