WOVEN

The paper proposes that the deficiencies of multimodal large models in spatial, embodied, physical, and temporal reasoning may all stem from the same missing capability—visual transition reasoning.

WOVEN is a data source and benchmark that annotates visual transition supervision along three dimensions—scene, action, and reasoning type—with data coming from rollouts of video pretrained generative models.

The final scale of WOVEN is 36,076 samples, 20 scene types, 5 action types, and 8 reasoning types.

Using generative model rollouts as a data source carries a risk of distribution shift: the generative model's own physical priors may contain biases or simplifications, making the data unable to fully reflect the complex dynamics of the real world.

In the WOVEN evaluation, even the strongest MLLM falls far below human level, and failure modes recur across different model families, so simply scaling up model size cannot eliminate these deficiencies.

Each WOVEN training subset of about 2,000 samples improves 22 of 26 external benchmarks in total, with a maximum improvement of 27.3 percentage points.

WOVEN data can replace 30%–50% of a task's own training data with comparable accuracy.

The training recipe suggested by the WOVEN paper: select data according to the reasoning operation taught by the supervision, rather than by action, scene, or domain; and prioritize samples with larger visual state changes to obtain better robustness.

WOVEN's argument essentially attributes part of MLLMs' deficiencies to a lack of world model capability—that is, although the model can describe image content, it cannot simulate how it evolves with actions.

If visual transition reasoning is indeed a transferable shared capability, then the training approach for multimodal world models needs to be adjusted: first build general modeling of visual state changes, then transfer to specific tasks, rather than piling up data separately for each downstream task.