[KongchangAI]
BenchmarkCV-Bench

CV-Bench

A benchmark for evaluating the spatial reasoning ability of vision-language models, containing questions about spatial relations such as left-right and above-below.

Timeline (last 90 days)

Nov 18

In the CV-Bench photo test, when reliable 2D cues are misleading, the accuracy of the three models (Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash) likewise drops.

Unverified50%
Nov 11

On the three models Qwen2.5-VL-7B, Qwen3-VL-4B and InternVL3-2B, after mirroring the ViT width-direction positional encoding, the proportion of the models reproducing the flipped-image answers on CV-Bench left/right questions reached 84.3%–93.2%.

Unverified50%
Nov 11

On CV-Bench questions with localized objects, random arrangement flips 18.9–31.0 percentage points more answers when reversing object order than when keeping the order.

Unverified50%

All Facts (3)

Source Articles