SpaceCast-Bench: A New Benchmark for Predictive Spatial Reasoning in Vision-Language Models

Spatial intelligence has long been a core challenge in embodied AI and robotics. Most existing spatial reasoning benchmarks remain at the level of "spatial perception"—recognizing relationships already visible in the input. But true spatial intelligence requires predictive spatial reasoning: building a scene from observations, anticipating the changes caused by interventions, and reasoning about unobserved outcomes. A new paper proposes SpaceCast-Bench, claimed to be the first benchmark that directly and diagnostically evaluates this capability.

From Perception to Prediction: Why a New Benchmark Is Needed

Traditional spatial reasoning benchmarks typically give a model an image and ask, "Is the cup to the left or right of the table?" Such questions test the model's ability to read visible relationships in a static image—this is at the perception level. However, spatial intelligence in the real world goes far beyond this. When you reach for a cup, you need to predict how much the cup will be occluded after your hand moves, how its position will change, and whether it will affect nearby objects.

The core contribution of SpaceCast-Bench is the observe-transform-infer framework. The model must first observe a real scene, then face some spatial intervention (e.g., object movement, viewpoint change), and finally answer questions about the transformed scene—which is not directly shown in the input.

This benchmark contains 3,862 questions from 182 real scenes, covering 16 task types, divided into three levels: static perception, local prediction, and global prediction. The higher the level, the greater the demands on scene understanding, spatial state updating, and relational reasoning about unobserved outcomes.

Vision-language models (VLMs) are the primary target of SpaceCast-Bench evaluation. These models combine a vision encoder (such as ViT) with a large language model, enabling the model to process both image and text inputs and output natural language answers. Typical representatives include GPT-4V, the Qwen-VL series, LLaVA, etc. VLMs excel at tasks such as image captioning and visual question answering, but their spatial understanding ability has long been debated—models may "guess" the right answer through language priors rather than actually building a spatial representation. The design of SpaceCast-Bench is precisely to strip away such shortcuts: since the transformed scene is not in the input, the model cannot answer by merely recognizing visible content; it must simulate the spatial transformation and reason about its outcome. This setting imposes purer requirements on the VLM's underlying spatial world model.

Performance of 21 Models: Far Below Human Level

The paper evaluated 21 models, and the results reveal a clear gap. The strongest model achieved only 58.0% accuracy, while human performance was 87.2%. Even more surprising, models specifically optimized for spatial tasks performed close to random guessing.

This shows that current vision-language models have a systematic shortcoming when handling "predictive spatial reasoning." Even if a model can accurately describe image content, once it is asked to simulate scene changes and reason about the resulting state, performance drops sharply.

Controlled experiments further found that bridge views are crucial for integrating dispersed observations. Bridge views refer to intermediate frames that connect different viewpoints or time points. Without these views, models struggle to relate multiple observations into a coherent spatial understanding.

Another interesting finding is that explicit 3D evidence (such as point clouds or depth maps) helps models more stably than generated result images or videos. This means that having the model "imagine" the transformed scene has limited effect, while providing structured three-dimensional information is more reliable.

Bridge views originate from multi-view geometry and robotic visual navigation. When an agent observes the same scene from different positions, adjacent viewpoints often have overlapping regions, and these overlapping frames serve as "bridges," helping the system establish correspondences between frames and thereby stitch together a coherent spatial representation. In human cognition, a similar mechanism is reflected in saccadic integration—we acquire multiple local views through eye movements, and the brain automatically fuses them into a unified spatial percept. Controlled experiments in SpaceCast-Bench show that removing bridge views significantly degrades model performance, indicating that current VLMs lack the ability to autonomously construct such cross-view correspondences and rely heavily on intermediate transition information already provided in the input.

How Much Can Fine-Tuning Improve Performance?

The paper also explored the effect of fine-tuning with procedurally generated data. On the Qwen3-VL-4B model, fine-tuning improved its accuracy from 34.0% to 65.7%, and achieved a macro-average gain on six out-of-domain benchmarks.

This result has two implications: on the one hand, predictive spatial reasoning ability can be significantly improved through targeted training; on the other hand, even after fine-tuning, 65.7% is still far from the human 87.2%, indicating that this task remains difficult for current architectures.

From an application perspective, this research has direct reference value for robot manipulation, autonomous driving, and AR/VR interaction. These scenarios all require systems not only to understand the current spatial state but also to predict the results after actions or environmental changes. SpaceCast-Bench provides a quantifiable diagnostic tool to help researchers locate specific weak links in the spatial reasoning chain.

Paper title: SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models

With the development of embodied intelligence, predictive spatial reasoning is likely to become one of the key indicators for measuring the practicality of vision-language models. The emergence of SpaceCast-Bench sets a new evaluation standard for this direction.

Procedurally generated data refers to training samples automatically synthesized through algorithms and rules, rather than manually annotated real data. In the fine-tuning experiments of SpaceCast-Bench, researchers used procedural methods to generate a large number of scene-question pairs with precise spatial transformation annotations, enabling the model to learn the patterns of "how the scene changes after an object moves." The advantages of this approach are that it can be scaled up大规模, annotations are precise, and costs are low, making it especially suitable for tasks such as spatial reasoning that are difficult to exhaustively annotate manually. However, procedural data also faces the domain gap problem: the textures, lighting, and object distributions of synthetic scenes differ from the real world, and models may learn superficial statistical patterns rather than generalizable spatial reasoning ability. This is also one of the potential reasons for the still-large performance gap after fine-tuning.

One-line Summary

New benchmark SpaceCast-Bench is the first to diagnostically evaluate predictive spatial reasoning; the strongest model reaches only 58%, far below humans at 87%.

Paragraph Summary

SpaceCast-Bench is the first benchmark to directly evaluate predictive spatial reasoning. It uses the observe-transform-infer framework, requiring models to observe a scene, anticipate changes caused by spatial interventions, and reason about unobserved outcomes. The benchmark contains 3,862 questions from 182 real scenes, covering 16 task types and divided into three levels: static perception, local prediction, and global prediction. Evaluation of 21 models shows that the strongest model reaches only 58.0% accuracy, while humans achieve 87.2%; spatial-specific models perform nearly at random. Controlled experiments find that bridge views are crucial for integrating dispersed observations, and explicit 3D evidence is more reliable than generated images or videos. Fine-tuning can improve Qwen3-VL-4B from 34.0% to 65.7%, but it remains far below human level.