How Can Sim-to-Real Testing Be Made More Realistic? The Pain Points of Humanoid Robot Deployment

In robotics, transferring policies trained in simulation to real hardware has always been a thorny problem. Recently, the co-founder of Sodero shared a demo of a humanoid robot walking while avoiding obstacles on Reddit, and raised a key question: can we make sim-to-real testing more realistic? This sparked a discussion about how to effectively validate policies before deployment.

When a Dance Policy Falls on Real Hardware

The founder had previously deployed a trained dance policy onto a Unitree G1 humanoid robot. According to his description, the policy ran "fairly well," but the robot sometimes fell. This phenomenon exposes the gap between simulation and reality—policies that perform well in simulation may fail on real hardware due to various unmodeled dynamics.

The causes of falling can be manifold: sensor noise, actuator delay, variations in ground friction, or communication latency in the control loop. These factors are often simplified or ignored in traditional simulation, causing policies to become "overconfident" in the virtual environment.

Unitree G1 is a mass-produced humanoid robot from Unitree, standing about 1.3 meters tall with 23 to 43 joint degrees of freedom (depending on configuration), equipped with depth cameras, IMUs, and force sensors, and focused on motion control and secondary development.

In sim-to-real research, low-cost, highly dynamic humanoid platforms like the G1 are representative: their joints use quasi-direct-drive actuators, which respond quickly but have non-negligible torque noise and delay, making them extremely sensitive to balance control. Researchers often use them as testbeds for validating reinforcement learning gait policies. Understanding the G1's hardware characteristics (such as control frequency, communication bus latency, joint friction, and backlash) helps explain why policies trained in simulation exhibit gait instability or even falls on real hardware.

System Simulation: Bringing the Control Chain into Simulation

The Sodero team is exploring an approach called "system simulation," which aims to make the simulation environment not only model physical laws but also simulate the robot's actual control system behavior. Specifically, they attempt to reproduce in simulation how the policy receives sensor data, how it sends control commands, and the delay and noise characteristics in these links.

The core of this idea is that an accurate physics engine alone is not enough—the realism of the control chain is equally important. If the sensor data in simulation is perfectly noise-free while the real IMU has drift, the robustness the policy learns in simulation may not be sufficient to cope with reality. By simulating communication latency, actuator response characteristics, and more, developers can identify problems the policy may encounter in real deployment earlier.

Sim-to-real refers to the process of deploying a policy trained in simulation onto real hardware, with the core challenge being the "reality gap"—the inability of simulation to fully reproduce real-world physics and control characteristics.

Common bridging methods include domain randomization (randomizing parameters such as friction, mass, and delay during training to improve robustness), system identification (calibrating simulation parameters with real-machine data), and the system-level simulation discussed in this article. Domain randomization lets the policy "see" a variety of possible real conditions, but if the randomization range does not match the real distribution, it can still fail. The value of system simulation lies in incorporating control-chain delay and noise into the simulation loop, complementing domain randomization to more accurately assess policy readiness.

Community Concerns: Which Differences Are Most Critical?

In the discussion, the founder asked the community: if you have transferred policies from simulation to real hardware, which differences caused the most trouble? What would you like simulation to reproduce more accurately? These questions strike at the core pain points of sim-to-real.

Common sources of discrepancy include inaccurate sensor noise models, simplified actuator dynamics, overly idealized ground contact models, and control frequency drops caused by computational latency. For humanoid robots, balance control is particularly sensitive to latency—a difference of a few milliseconds can cause gait instability. Therefore, if system simulation can capture these details, it will help researchers filter out more reliable policies before hardware testing.

Actuator dynamics refers to the actual response characteristics of the motor, reducer, and driver combination after receiving a command, including torque-speed curves, friction, backlash, saturation limits, and response delay. In simulation, actuators are often simplified as ideal torque sources or first-order inertial elements, ignoring these nonlinearities.

For humanoid robots, actuator dynamics directly affect joint tracking accuracy and balance recovery capability: for example, reducer backlash creates a "dead zone" for small commands, and torque saturation limits the ability to correct before a fall. If simulation does not model these characteristics, the policy may learn control laws that rely on ideal actuators, manifesting on real hardware as jitter, delayed response, or gait instability. Therefore, an accurate actuator model is one of the key elements for improving sim-to-real success rates.

What More Realistic Testing Means

The ultimate goal of making simulation more realistic is to reduce the risk and cost of real-hardware testing. Exposing the failure boundaries of a policy in simulation can guide developers to conduct targeted hardware debugging rather than blindly trial-and-error on real hardware. Sodero's exploration represents a trend: moving from purely physical simulation toward system-level simulation that includes the control chain, thereby assessing policy readiness more comprehensively.

Of course, this does not mean simulation can completely replace real-hardware testing. The complexity and unpredictability of the real world always exist, but more realistic simulation can at least give us a clearer understanding of a policy's strengths and weaknesses before we step into the lab.