[KongchangAI]
· 2 min read· 1,090 words

Simulation Training Enables a 27B Open-Source Model to Natively Control Real Robots

Simulation Training Enables a 27B Open-Source Model to Natively Control Real Robots

A 27B open-source model can natively control real robots after pure-simulation RL with zero demonstration data.

The study proposes the PEAR post-training engine, which uses Robot-use Reinforcement Learning (RURL) to train a 27B open-source reasoning model into a robot-use agent that directly outputs end-effector poses, entirely in simulation with zero real demonstrations. The trained model reaches 60% success on a real dual-arm robot (vs. zero for the base model) and lifts LIBERO success from 1.6% to 74.4%, approaching frontier closed-source models, while transferring to unseen robots and new instructions. Compared with the demonstration-fine-tuned VLA model π0.5, it is far more robust when tasks are swapped (57.4% vs. 0.75% on LIBERO-Pro), and training causes no degradation on nine general benchmarks, validating robot-use RL as a viable base-model post-training component.

Robot-use agents (RUA) are seen as an important path toward general-purpose robot control, but until now only frontier closed-source models like GPT-6 Astra and Claude Fable 5.2 have demonstrated strong capabilities, while open-weight models have consistently lagged behind. A new study attempts to break this pattern: under fully simulated conditions with zero demonstration data, it uses reinforcement learning to train a 27B open-source reasoning model into an agent capable of natively controlling real robots.

Simulation training enables an open-source model to natively control real robots

From Computer-Use Agents to Robot-Use Agents

The core idea of this work draws on the established paradigm of computer-use agents (CUA). A CUA observes the screen, reasons in a chain of thought, and then outputs the next action; correspondingly, a robot-use agent observes the environment, performs chain-of-thought reasoning, and directly generates the next end-effector pose—with no additional execution harness in between.

To this end, the research team built a post-training engine called PEAR (Post-training Engine for Agentic Robotics) and trained the model on large-scale simulated robot tasks using "Robot-use reinforcement learning" (RURL)—a pipeline highly similar to how frontier models acquire computer-use capabilities. The key difference: the entire training process takes place entirely in simulation and does not rely on any real-world demonstration data.

Computer-use agents (CUA) are a class of agents that let large models directly operate computer interfaces: the model receives observations such as screenshots and accessibility trees, plans steps in a chain of thought, and then outputs atomic actions such as clicks, typing, and scrolling. Its key characteristic is being "end-to-end"—the model itself is the controller, requiring no external scripting or tool-orchestration layer. By analogy, a robot-use agent's observations are camera images and proprioceptive state, and its action space is the six-degree-of-freedom pose of the end effector (possibly plus gripper open/close). The difficulty in transferring the CUA paradigm to robots lies in: actions change from discrete interface events into continuous physical quantities, and the model must handle three-dimensional space, temporal dynamics, and contact physics; meanwhile, robot data is far scarcer than web interaction data. This is precisely the context behind the article's emphasis on "simulation reinforcement learning + zero demonstrations."

RL post-training refers to continuing to optimize model parameters on top of a pretrained base model using reward signals rather than supervised labels. Unlike supervised fine-tuning (SFT), it does not require demonstrations of "correct actions," only a reward function that can evaluate whether outcomes are good or bad, making it particularly suited to scenarios with scarce demonstration data. In robot simulation, rewards can be computed automatically from task completion, object pose error, and the like, and combined with large-scale parallel simulation environments can generate massive amounts of interaction experience. RURL in the article applies this pipeline to robot manipulation tasks: the model repeatedly trials and errors in the environment, improving its success rate through policy-gradient-style algorithms. This also explains why the base model's success rate is zero while post-training reaches 74.4%—RL can "activate" the latent spatial reasoning ability in the pretrained model into an executable manipulation policy.

Training Results: Simulated Skills Transfer to the Real World

The most noteworthy finding is that skills acquired in simulation generalize remarkably well: they can not only complete the training tasks but also transfer to unseen robot embodiments, transfer to the real world, and respond to entirely new instructions.

After training in the LIBERO and RoboDojo simulation environments, the model achieved a 60% trial success rate on a real dual-arm robot, whereas the base model never succeeded even once. On the LIBERO benchmark, training raised the success rate from 1.6% to 74.4%, approaching the level of GPT-6 Astra and GPT-6.1 Sol at maximum inference intensity.

The significance of this result is that open-source models need not rely on expensive real-robot data collection—simulation reinforcement learning alone may be enough to obtain usable real-robot manipulation capabilities.

Comparison with Vision-Language-Action Models

The research team also compared the model with $\pi_{0.5}$. $\pi_{0.5}$ is a vision-language-action model (VLA) fine-tuned on demonstration data, representing the current mainstream approach to robot policy learning.

The comparison reveals the essential difference between the two approaches. When the task itself changes, the model is significantly more robust. On the LIBERO-Pro benchmark, when the scene is familiar but the task is replaced with a new one, $\pi_{0.5}$'s success rate is only 0.75%, while the model reaches 57.4%. This shows that demonstration-fine-tuned VLA models tend to overfit to specific tasks, whereas reasoning models trained with reinforcement learning possess stronger task-level adaptability.

Vision-language-action models (VLA) are the current mainstream paradigm for robot policy learning: using a vision-language model as the backbone, adding an action output head on top, and then fine-tuning with behavior cloning on large-scale robot demonstration data. Representative works include RT-2, OpenVLA, and the π series in this article. VLA's advantages are stable training and the ability to directly inherit visual-semantic knowledge; its limitation is that behavior cloning is essentially imitation, and the policy distribution is constrained by the tasks covered in the demonstration data, making it prone to failure when task goals or scene structure change—precisely the overfitting phenomenon reflected by the 0.75% success rate on LIBERO-Pro in the article. In contrast, the reinforcement-learning-trained reasoning model re-understands and re-plans the task at every step, with actions driven by chain-of-thought rather than produced by a fixed mapping, and therefore has greater room for adaptation when tasks are replaced.

General Capabilities Do Not Degrade

Whether robot manipulation training harms the model's original reasoning and understanding capabilities is a common concern for this kind of post-training. The research's answer is no: the trained model showed no measurable performance degradation on nine reasoning, coding, and vision benchmarks.

This means that robot-use reinforcement learning can serve as a post-training component for base models that coexists with existing capabilities, rather than trading general intelligence for robot skills.

Implications for Open-Source Robot Agents

This study sends several clear signals. First, robot-use agents need not be the exclusive capability of closed-source frontier models; open-weight models can reach comparable levels through an appropriate post-training pipeline. Second, the value of simulation reinforcement learning in robotics is further validated—it is not just a means of reducing data costs but an effective path to acquiring transferable skills. Third, the architecture of chain-of-thought reasoning plus direct end-effector pose output gives the model stronger adaptability than traditional VLA when tasks change.

Of course, a 60% real-robot success rate is still far from reliable deployment, and the sim-to-real gap remains. But as a direction for base-model post-training, robot-use reinforcement learning has already demonstrated potential worth continued investment.

#robot-use agent#reinforcement learning#open-source model#sim-to-real#VLA
Share:

Related articles