Humanoid Robots Walking on Sand: Fine-Tuning Policies via Scanned Real Terrain to Solve Sim-to-Real Physics Calibration Challenges

From Falling to Standing: A Sand Fine-Tuning Experiment

A developer used an iPhone LiDAR to scan outdoor sand, reconstructed it into a simulation environment, and then tested a pretrained locomotion policy for the Unitree G1 humanoid robot. The original policy fell immediately on the sand, but after fine-tuning with randomized starting points, the robot completed the full trial and remained upright. This result intuitively demonstrates the value of terrain fine-tuning, but it also exposes a more thorny problem: geometry can be scanned, but physical properties are hard to replicate.

According to the Reddit post, the team's current uncertainty centers on physical parameters rather than the geometric model—"ground that looks the same can slide or sink in completely different ways depending on moisture and load," and contact parameters still rely on manual tuning.

Geometry Is Easy, Physics Is Hard

Scanning terrain with LiDAR and reconstructing meshes is relatively mature engineering. The iPhone's LiDAR sensor can provide point clouds of sufficient precision, and with reconstruction tools it can generate terrain assets usable for simulation. The problem is that contact models in simulators require more than just shape.

Deformable ground like sand involves multiple variables: friction coefficients, degree of granular compaction, the effect of moisture content on cohesion, and sinkage depth under load. The same patch of sand in dry versus wet conditions can produce completely different force responses at the robot's feet. Manual tuning can get a single trial to work, but it is hard to guarantee the policy's generalization ability in the real world.

Domain Randomization is a core training technique in sim-to-real transfer. The basic idea: during simulation training, apply random perturbations to physical and visual parameters such as lighting, texture, friction coefficients, mass, and sensor noise, so that the policy learns across a large number of "simulation variants." In this way, the real world is treated as one sample point in the distribution of simulation parameters, and the policy gains generalization ability by having seen enough variation. The effectiveness of domain randomization depends heavily on the setting of the randomization ranges: too narrow and it fails to cover real-world differences; too wide and the policy becomes conservative and performance degrades. In practice, curriculum-based expansion or narrowing ranges based on real data is often used to balance robustness and performance.

System Identification instead collects input-output data from the real system and infers unknown parameters of the simulation model (such as friction coefficients, joint damping, and contact stiffness). In robotics, a common approach is to have the robot perform controlled excitation motions, record joint torques and motion responses, and then optimize so that simulated trajectories align with measured trajectories. Compared with domain randomization, system identification can yield point estimates of parameters closer to reality, but it relies on high-quality measurement data and additional experimental procedures, and its effectiveness is limited for parameters that are difficult to measure directly (such as sand moisture content).

Calibration Paths the Community Is Watching

The core question the post raises to the community is: in sim-to-real for outdoor terrain, how do you calibrate friction and deformable ground models? The poster lists several candidate paths—large-range domain randomization, collecting measurement data from the real robot, or other approaches—and hopes for reproducible papers or apparatus references.

From public discussion, domain randomization is currently the most commonly used fallback: apply wide-range random perturbations to parameters such as friction coefficients and ground stiffness in simulation, forcing the policy to learn to remain stable under uncertain physical conditions. But the problem with this approach is also obvious—the wider the randomization range, the more conservative the policy, potentially sacrificing performance on real ground.

Another path is to use real robot data for system identification. Have the robot perform controlled motions on known terrain, and infer contact parameters from foot force sensors and proprioceptive data. This is closer to reality than pure randomization, but requires additional hardware and experimental procedures.

The Peculiarity of Deformable Ground

Sand differs from rigid ground: the contact model must account for the yield behavior of granular media. A common simplification is to approximate it with a deformable terrain model or an equivalent spring-damper system, but parameter calibration remains the bottleneck. Moisture is an especially tricky variable—it affects both friction and load-bearing capacity, and scan data cannot capture it directly.

There is reason to believe a hybrid approach may be more practical: first use domain randomization to cover parameter uncertainty, then use a small amount of real measurement data to narrow the ranges of key parameters. This preserves the policy's robustness without losing performance due to excessive randomization.

Deformable terrain models are a common means for simulators to handle non-rigid ground such as sand, mud, and snow. Take MuJoCo's soft contact model as an example: it introduces equivalent spring-damper-friction elements at ground contact points, approximating the load-bearing and yield behavior of granular media by adjusting stiffness, damping, friction coefficients, and slip parameters. Such models are computationally efficient, but their parameters lack clear physical meaning and often need to be calibrated separately for each ground material. More refined methods include discrete element method (DEM)-based granular simulation or position-based dynamics (PBD) sand models, which can capture granular flow and compaction, but are computationally expensive and difficult to use for large-scale reinforcement learning training. Therefore, in practice, simplified models plus domain randomization are often used to cover model error, and calibrating the parameters of these simplified models is precisely the current bottleneck.

Implications for Sim-to-Real Practice

The value of this case is that it clearly delineates the boundary of sim-to-real: geometric fidelity is no longer the main obstacle; physical fidelity is. Fine-tuning can make a policy stand on a specific scanned terrain, but for a policy to walk reliably on real sand, the calibration method for contact parameters still needs a more systematic solution.

For teams working on outdoor terrain robotics, directions worth watching include: low-cost measurement pipelines for real terrain parameters, efficient simulation models for deformable ground, and strategies combining domain randomization with system identification. The post itself does not yet provide answers, but the question it raises—how to validate terrain physics in simulation—is precisely one of the core gaps in current sim-to-real research.