WorldGuide: Making Video World Models Actually Execute Procedural Tasks

The Next Frontier of Video Generation: From "Looks Right" to "Actually Done"

Video generation models can already synthesize realistic visual trajectories, but a fundamental question is emerging: when a task requires dozens of steps and takes minutes to complete, can the model decide what to do next based on what it has already generated?

This sounds like a planning problem, but it is actually a rift between generation and execution. Recent research on WorldGuide points out that existing systems either adopt open-loop generation—unable to adjust subsequent actions based on execution results—or rely on pretrained executors or indirect verification—creating a gap between "deciding an action" and "realizing an action."

WorldGuide's core claim is that procedural video generation should essentially be closed-loop task execution in visual world space. Given an initial image and a task goal, the model needs to predict atomic actions, generate corresponding video clips, and then use the generated results to select the next action or determine task completion.

Closed-Loop Execution: Planner and Executor Share the Same Training Signal

WorldGuide's architecture revolves around two core components. The Planner learns from visual progress to predict the next atomic action or judge whether the task is complete; the Executor is directly trained to convert predicted actions into video clips.

The key is that these two modules are jointly trained on the same step-level procedural demonstration data. This is fundamentally different from approaches that rely on external pretrained executors: WorldGuide's executor itself is purpose-built for this planning signal.

State maintenance during long-horizon execution is handled by hierarchical visual memory. This design preserves historical information while controlling token cost—for tasks requiring hundreds of operations, this is a prerequisite for feasibility.

Why Open-Loop Approaches Fall Short

The problem with open-loop generation is that it assumes each generated step will "automatically connect." But in real procedural tasks, subtle deviations in earlier steps gradually accumulate, causing subsequent actions to drift completely off course. WorldGuide's closed-loop design lets the model "see" its own generated results at each step and adjust accordingly—this is precisely why it can open up a gap in task success rates.

The difference between open-loop and closed-loop control systems can be understood through a simple analogy: open-loop is like following a pre-written script, where nothing that happens midway changes subsequent steps; closed-loop feeds the actual result of each step back, and adjusts the next step accordingly. In classical automatic control theory, closed-loop systems are generally more stable and precise than open-loop systems because they have error feedback. Video generation has long been an open-loop paradigm: given an image and a goal, generate the complete video all at once (or segment by segment according to a fixed script). But in procedural tasks, each generated video segment may differ subtly from the ideal state. An open-loop system has no way to perceive these differences, and deviations amplify level by level along the timeline, ultimately causing task failure. WorldGuide's closed-loop idea is precisely to treat the "generated result" as a feedback signal, letting the planner re-evaluate the visual state at each step and decide the next action.

An atomic action refers to an irreducible minimal unit of operation with clear semantics, such as "pick up the cup," "open the drawer," or "turn the knob." Procedural tasks usually consist of a sequence of atomic actions in a specific order. The emphasis on "atomic" is because it is both small enough for the video model to generate short clips and semantically clear enough for the planner to make sequential decisions. If the action granularity is too coarse, the executor may fail to faithfully realize it; if too fine, the planner must undertake lengthy sequential reasoning, which easily leads to semantic drift. What WorldGuide Bench annotates for each data point is exactly "which atomic action should be executed under the current visual progress." This layer of supervision enables the planner to learn a mapping from visual state to action decision, and also explains why matching atomic action granularity with task complexity will become a focus of subsequent research.

WorldGuide Bench: Filling the Gap in Step-Level Supervision

Training a joint planner-executor model faces a practical obstacle: the lack of step-level action-video supervision data. Existing video datasets usually provide only complete videos and task labels, lacking fine-grained annotations of "what atomic action should be executed in a given visual state."

WorldGuide Bench was constructed precisely to solve this problem. It contains approximately 59K step-annotated videos, covering 245 tasks and 27 procedure categories. Each data point is annotated with the atomic action corresponding to the current visual progress, enabling the planner to learn a mapping from visual state to action decision.

The scale of this dataset is considerable among comparable works, and it also provides a reusable evaluation benchmark for subsequent research.

In the context of video generation, a world model refers to the model's ability to predict "how the world will change after the next interaction." Traditional video world models aim to generate future frames that are visually plausible and temporally coherent; this paper pushes further toward "operation completion in visual space"—not only predicting "what will be seen next," but also deciding "what should be done next to actually advance the task." This effectively brings video world models closer to task and motion planning (TAMP) in robotics: both require long-horizon decision-making under partial observability. WorldGuide's value lies in using pure visual generation to bypass explicit state estimation and action primitive definitions, allowing the model to execute in a closed loop directly in pixel space, thereby providing a new paradigm reference for directions such as robotic manipulation and instructional video generation.

Experimental Results: Coupling Planning and Execution Brings Substantial Gains

On WorldGuide-Bench, WorldGuide achieved a 33.33% task success rate. The comparison target is the recently strong video model MiniMax-H3, which achieved a success rate of 29.90% when receiving a reference action plan.

Interestingly, MiniMax-H3 received an additional action plan input in this comparison, while WorldGuide did not—this further indicates that merely providing action planning information is not enough to guarantee task completion; whether the executor can faithfully realize the plan is the real bottleneck.

On VideoCraft-Bench, the gap is even more pronounced: WorldGuide reaches a 47.69% success rate, while MiniMax-H3 achieves only 32.73% when given only the goal condition. The results on both benchmarks consistently point to the same conclusion: coupling planning with a learnable executor is key to goal-directed procedural video generation.

What This Means

WorldGuide's significance is not only in the numbers. It redefines the task boundary of video world models—no longer merely "generating plausible visual content," but "completing operations in visual space." The impact of this perspective shift may extend to robotic manipulation, instructional video generation, interactive content creation, and many other directions.

Of course, a 33.33% task success rate also shows that the problem is far from solved. The efficiency of long-horizon visual memory, the matching between atomic action granularity and task complexity, and cross-category generalization are all directions that subsequent work needs to continue tackling. But at least, WorldGuide demonstrates the feasibility of a closed-loop planning-execution architecture in procedural video tasks, and provides a clear evaluation benchmark for tracking progress.

One-line Summary

WorldGuide reframes procedural video generation as closed-loop task execution in visual space, with joint training of planning and execution significantly improving task success rates.

Paragraph Summary

WorldGuide points out that existing video generation systems have a rift between generation and execution in long-horizon procedural tasks: open-loop approaches cannot adjust subsequent actions based on execution results, while approaches relying on external executors separate decision-making from realization. To address this, WorldGuide proposes a closed-loop execution architecture, in which a Planner predicts the next atomic action or judges completion, and an Executor directly generates the corresponding video clip. The two are jointly trained on the same step-level demonstration data, and hierarchical visual memory maintains long-horizon state while controlling token cost. To support training, the authors built WorldGuide Bench, containing about 59K step-annotated videos, 245 tasks, and 27 procedure categories. On WorldGuide-Bench, the task success rate reaches 33.33%, surpassing MiniMax-H3 with an action plan (29.90%); on VideoCraft-Bench under the goal condition, it reaches 47.69%, far exceeding the latter's 32.73%.