World Action Model
A model type that converges world models with policy learning: while predicting environment dynamics, it incorporates actions as conditioning or output variables into the modeling, giving the model a closed-loop capability to 'imagine consequences and select actions.' It descends from learned dynamics models in model-based reinforcement learning.
Core Facts
Timeline (last 90 days)
World action models jointly predict video and actions; the benefits are higher sample efficiency, greater stability on long-horizon tasks, and easier transfer to different embodiments; the cost is that training requires large amounts of video data with action annotations, and inference compute overhead is also greater.
Models such as world action models typically rely on massive amounts of human video or simulation data for pre-training, and are then fine-tuned with a small amount of robot demonstration data.
Praxis-1 is positioned as a "world action model", a type of model that not only understands the visual world but can also directly output physical actions to drive robots.
In WAM-style methods, "predicting well" and "acting correctly" are not equivalent, and the representations on which action generation depends need separate constraints.
WAMs have a hidden pitfall: the latent variables needed to predict the entire scene and the latent variables that truly drive actions are often mixed together.
World Action Models (WAMs) couple predictive world modeling with action generation, enabling the model not only to predict the next state of the environment but also to guide agents in taking their next actions accordingly.
Valerant leverages a pre-trained action-conditioned world model to predict subsequent visual frames based on given actions.
All Facts (7)
World Action Models (WAMs) couple predictive world modeling with action generation, enabling the model not only to predict the next state of the environment but also to guide agents in taking their next actions accordingly.
75%UnverifiedWorld action models jointly predict video and actions; the benefits are higher sample efficiency, greater stability on long-horizon tasks, and easier transfer to different embodiments; the cost is that training requires large amounts of video data with action annotations, and inference compute overhead is also greater.
50%UnverifiedPraxis-1 is positioned as a "world action model", a type of model that not only understands the visual world but can also directly output physical actions to drive robots.
50%UnverifiedModels such as world action models typically rely on massive amounts of human video or simulation data for pre-training, and are then fine-tuned with a small amount of robot demonstration data.
50%UnverifiedIn WAM-style methods, "predicting well" and "acting correctly" are not equivalent, and the representations on which action generation depends need separate constraints.
50%UnverifiedWAMs have a hidden pitfall: the latent variables needed to predict the entire scene and the latent variables that truly drive actions are often mixed together.
50%UnverifiedValerant leverages a pre-trained action-conditioned world model to predict subsequent visual frames based on given actions.
50%