ThinkWorld
A reinforcement learning VLA framework based on world imagination that models action reasoning as a Chain-of-Dynamics and uses videos without action annotations for cross-embodiment skill transfer.
Timeline (last 90 days)
ThinkWorld is a VLA learning framework that grounds action reasoning in world dynamics and enhances reasoning through world-imagination reinforcement learning.
ThinkWorld's core innovation lies in representing each action thought as a Chain-of-Dynamics (CoD), that is, a sequence of discrete motion tokens that captures the intermediate dynamics from the current world to a future state and serves as a general action representation across embodiments.
ThinkWorld's training process consists of two steps: first, the CoD generation capability is initialized with action-annotated robot demonstration data; then the model predicts, for each CoD, the future state it induces, compares the prediction with the real observation, and uses reinforcement learning to increase the generation probability of those CoDs whose predictions are more accurate.
ThinkWorld's "world imagination" mechanism allows videos without action annotations to provide supervision signals through state transitions, thereby incorporating human videos into training in the same reasoning space.
Experiments in simulated environments and on real robots show that ThinkWorld can effectively achieve skill transfer from human videos to robots and exhibits strong generalization to unseen tasks and environments.
Reinforcing, in ThinkWorld, those CoDs whose predictions match the actual observations more closely is typically achieved through policy optimization in reinforcement learning, for example by using the negative of the prediction error as the reward and adopting REINFORCE or preference-optimization-style objectives to increase the generation probability of high-reward sequences.
ThinkWorld provides a unified framework in which human and robot experience reinforce each other within the same chain-of-dynamics reasoning space, promising to alleviate VLA models' dependence on action-annotated data.
All Facts (7)
ThinkWorld is a VLA learning framework that grounds action reasoning in world dynamics and enhances reasoning through world-imagination reinforcement learning.
50%UnverifiedThinkWorld's core innovation lies in representing each action thought as a Chain-of-Dynamics (CoD), that is, a sequence of discrete motion tokens that captures the intermediate dynamics from the current world to a future state and serves as a general action representation across embodiments.
50%UnverifiedThinkWorld's training process consists of two steps: first, the CoD generation capability is initialized with action-annotated robot demonstration data; then the model predicts, for each CoD, the future state it induces, compares the prediction with the real observation, and uses reinforcement learning to increase the generation probability of those CoDs whose predictions are more accurate.
50%UnverifiedThinkWorld's "world imagination" mechanism allows videos without action annotations to provide supervision signals through state transitions, thereby incorporating human videos into training in the same reasoning space.
50%UnverifiedExperiments in simulated environments and on real robots show that ThinkWorld can effectively achieve skill transfer from human videos to robots and exhibits strong generalization to unseen tasks and environments.
50%UnverifiedReinforcing, in ThinkWorld, those CoDs whose predictions match the actual observations more closely is typically achieved through policy optimization in reinforcement learning, for example by using the negative of the prediction error as the reward and adopting REINFORCE or preference-optimization-style objectives to increase the generation probability of high-reward sequences.
50%UnverifiedThinkWorld provides a unified framework in which human and robot experience reinforce each other within the same chain-of-dynamics reasoning space, promising to alleviate VLA models' dependence on action-annotated data.
50%