[KongchangAI]
· 2 min read· 1,004 words

MBZUAI Open-Sources PAN 2: A 29B Physical AI World Model

MBZUAI Open-Sources PAN 2: A 29B Physical AI World Model

MBZUAI open-sources the 29B world model PAN 2, reasoning out 20-second action futures via video.

MBZUAI's Institute of Foundation Models open-sourced PAN 2, a world model with an LLM planner plus 29B diffusion Transformer that renders tasks into video futures up to ~20 seconds. Its value lies in a locally deployable imagination sandbox for physical AI trial and error; Apache 2.0 signals commercial intent. Yet the 20s window, video realism vs. physical consistency, and cascading planner-generator error remain open boundaries.

MBZUAI IFM releases PAN 2 world model

Physical AI has long been caught in a contradiction: robots need to learn through trial and error in the real world, but the real world is both expensive and dangerous. World models offer a workaround—letting agents first reason through consequences in "imagination." PAN 2, recently open-sourced by the MBZUAI Institute of Foundation Models (IFM), is a noteworthy engineering sample along this path.

What PAN 2 Is: A Dual-Core Structure of Planner and Generator

PAN 2's architecture can be split into two halves. The first half is an LLM-based planner responsible for decomposing tasks into executable action sequences; the second half is a 29B-parameter diffusion Transformer responsible for "rendering" these abstract actions into video-form future frames. Connected in series, the model can simulate action outcomes lasting up to approximately 20 seconds.

This division of labor—"plan first, then imagine"—is not new, but PAN 2 pushes the scale to 29B and releases it fully under the Apache 2.0 license. For physical AI research, a freely modifiable, locally deployable world model foundation holds more long-term value than a black box accessible only through an API.

Why World Models Matter for Physical AI

Traditional robot training relies on reinforcement learning or demonstration data in real environments, which is costly to collect and limited in scenario coverage. The core premise of world models is: if a model can accurately predict "what the world will look like after executing a certain action," then policies can perform low-cost trial and error inside the model.

PAN 2 uses video as its prediction medium, meaning its output is not just state vectors but complete scene evolution with spatial structure, object motion, and temporal information. The 20-second reasoning window is sufficient to cover the full loop of common manipulation tasks such as grasping, moving, and placing, giving it practical potential as an "imagination sandbox."

Diffusion Transformer (DiT) is the core architecture of PAN 2's generator component. It combines the denoising process of traditional diffusion models with the sequence modeling capability of Transformers: diffusion models generate data from random tensors by gradually removing noise, while the Transformer predicts the noise at each step and leverages self-attention mechanisms to capture long-range spatiotemporal dependencies between video frames. Compared to earlier U-Net-based diffusion architectures, DiT exhibits better performance scalability as parameter count grows, which is the key technical foundation enabling PAN 2 to push its generator to 29B parameters. Understanding this helps grasp the capability boundaries and scaling logic of world models on video prediction tasks.

Cascading error refers to the phenomenon in multi-stage systems where deviations in the output of an earlier stage are inherited and amplified by subsequent stages. In PAN 2's two-stage structure, the LLM planner first outputs an action sequence, and the diffusion generator then reasons out video based on it. If the planner's understanding of the task has a slight deviation, the generator will faithfully "imagine" the future of a wrong action sequence, and the video reasoning itself may also introduce physical inconsistencies. After the two levels of error compound, the final simulation result may significantly deviate from the causal chain of the real world. This is why long-horizon tasks require segmented reasoning, and each segment's error accumulates into the next—cascading error is an important metric for evaluating the practical utility of world models.

The Significance of Open Source and Apache 2.0

Choosing Apache 2.0 rather than a research-only license is a clear signal from PAN 2: IFM wants it to enter commercial and engineering practice, not just remain at the level of paper reproduction. The 29B scale imposes certain compute requirements, but it falls within the accessible range among open-source world models—research teams and companies with reasonable infrastructure can fine-tune and adapt it to their domains.

This also echoes a current trend in the physical AI field—the foundation model layer is moving from closed to open, and the focus of competition is gradually shifting from "whose model is stronger" to "whose ecosystem is more active."

Boundaries That Warrant a Measured View

PAN 2's 20-second simulation window is the practical boundary of the current architecture; longer tasks require segmented reasoning, and errors accumulate along the chain. There is also tension between video generation quality and physical consistency: photorealistic visuals do not equal correct dynamics, and the latter is precisely the key to transferring robot policies to the real world.

Furthermore, while the two-stage structure of planner and generator is clearly decoupled, it introduces cascading error—once planning deviates, no matter how precise the subsequent video reasoning is, it cannot compensate. These issues are not unique to PAN 2 but are challenges the entire world model approach must face collectively.

PAN 2's value lies not in having solved all the problems of physical AI, but in providing an experimental platform that is large enough, open enough, and clearly structured. For a field still searching for its paradigm, such a foundation is itself a form of progress.

One-line Summary

MBZUAI open-sources the 29B-parameter world model PAN 2, using video to reason out 20-second action futures.

Paragraph Summary

MBZUAI's Institute of Foundation Models has open-sourced the world model PAN 2, employing a dual-core architecture of an LLM planner plus a 29B diffusion Transformer that can decompose tasks into action sequences and render them into video futures lasting up to approximately 20 seconds. The article notes that the core value of world models lies in enabling robot policies to perform low-cost trial and error in an "imagination sandbox," and PAN 2 uses video as its medium to preserve spatial structure and temporal information, with a 20-second window sufficient to cover the full loop of common manipulation tasks such as grasping, moving, and placing. The Apache 2.0 open-source license sends a clear signal of entry into commercial and engineering practice. At the same time, the article soberly points out that the 20-second window, the tension between video realism and physical consistency, and the cascading error between planner and generator are boundaries the current approach must collectively confront.

#MBZUAI#PAN 2#world model#physical AI#diffusion Transformer
Share:

Related articles