CueClash
A benchmark for evaluating the three-dimensional spatial reasoning ability of vision-language models. By rendering scenes, it separates 2D picture-plane cues from the true 3D layout and quantifies the degree to which models rely on 2D shortcuts.
Timeline (last 90 days)
The new benchmark CueClash renders scenes so that 2D cues for five spatial relations are correct in exactly half of the statements and wrong in the other half, thereby quantifying vision-language models' reliance on 2D shortcuts.
The researchers define the "shortcut gap" as accuracy when 2D cues are correct minus accuracy when the cues are wrong.
For Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash (low reasoning effort), the shortcut gap reaches as high as +18 to +37 percentage points.
Even after excluding two relations confounded by occlusion or boundary cases, the shortcut gap remains +7 to +29 percentage points.
When given ground coordinates and relation definitions directly, the three models (Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash) reach 99% to 100% answer accuracy.
CueClash's findings show that even the most advanced models still exhibit significant shortcut reliance at low reasoning effort, and that genuine 3D understanding requires dedicated architectures and training strategies rather than merely scaling up parameters.
All Facts (6)
The new benchmark CueClash renders scenes so that 2D cues for five spatial relations are correct in exactly half of the statements and wrong in the other half, thereby quantifying vision-language models' reliance on 2D shortcuts.
50%UnverifiedThe researchers define the "shortcut gap" as accuracy when 2D cues are correct minus accuracy when the cues are wrong.
50%UnverifiedFor Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash (low reasoning effort), the shortcut gap reaches as high as +18 to +37 percentage points.
50%UnverifiedEven after excluding two relations confounded by occlusion or boundary cases, the shortcut gap remains +7 to +29 percentage points.
50%UnverifiedWhen given ground coordinates and relation definitions directly, the three models (Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash) reach 99% to 100% answer accuracy.
50%UnverifiedCueClash's findings show that even the most advanced models still exhibit significant shortcut reliance at low reasoning effort, and that genuine 3D understanding requires dedicated architectures and training strategies rather than merely scaling up parameters.
50%