ReGap-Bench
A paired benchmark for studying the referential gap in multimodal reasoning. It contains five categories of visual reasoning tasks and measures the effect of reference formulation on reasoning accuracy by controlling reference-style variables.
Timeline (last 90 days)
The researchers built ReGap-Bench, a paired benchmark containing five types of visual reasoning tasks, to test the effect of external reference on model performance.
In the design of ReGap-Bench, the scene, question, and answer of the same problem are completely fixed, and the only thing that changes is the reference style, thereby isolating the effect of the reference variable.
Experimental results show that task-matched reference styles can improve accuracy by up to 25%.
All Facts (3)
The researchers built ReGap-Bench, a paired benchmark containing five types of visual reasoning tasks, to test the effect of external reference on model performance.
50%UnverifiedIn the design of ReGap-Bench, the scene, question, and answer of the same problem are completely fixed, and the only thing that changes is the reference style, thereby isolating the effect of the reference variable.
50%UnverifiedExperimental results show that task-matched reference styles can improve accuracy by up to 25%.
50%