[KongchangAI]
BenchmarkCueClash基准

CueClash

A benchmark for evaluating the three-dimensional spatial reasoning ability of vision-language models. By rendering scenes, it separates 2D picture-plane cues from the true 3D layout and quantifies the degree to which models rely on 2D shortcuts.

Timeline (last 90 days)

Nov 18

The new benchmark CueClash renders scenes so that 2D cues for five spatial relations are correct in exactly half of the statements and wrong in the other half, thereby quantifying vision-language models' reliance on 2D shortcuts.

Unverified50%
Nov 18

The researchers define the "shortcut gap" as accuracy when 2D cues are correct minus accuracy when the cues are wrong.

Unverified50%
Nov 18

For Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash (low reasoning effort), the shortcut gap reaches as high as +18 to +37 percentage points.

Unverified50%
Nov 18

Even after excluding two relations confounded by occlusion or boundary cases, the shortcut gap remains +7 to +29 percentage points.

Unverified50%
Nov 18

When given ground coordinates and relation definitions directly, the three models (Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash) reach 99% to 100% answer accuracy.

Unverified50%
Nov 18

CueClash's findings show that even the most advanced models still exhibit significant shortcut reliance at low reasoning effort, and that genuine 3D understanding requires dedicated architectures and training strategies rather than merely scaling up parameters.

Unverified50%

All Facts (6)

Source Articles