Recreating a Real Bubble Tea Station in Isaac Sim: Script-Driven Unitree G1 Brews Tea

Bringing real-world scenarios into physics simulation environments and having robots complete real tasks within them has been a prominent idea in robot learning in recent years. Recently, developers shared their experiment on Reddit: they recreated a self-service bubble tea station in NVIDIA Isaac Sim and had a Unitree G1 humanoid robot complete a five-step tea-brewing process via scripting.

From Real Scene to Digital Twin

The research team has long been working on one thing—transforming real-world locations into physically interactive 3D environments. This time, the subject was a self-service bubble tea station. They built a "digital cousin" of this tea station in Isaac Sim, placed the Unitree G1 into the scene, and wrote a five-step tea-brewing script for the robot.

It should be noted that the human video on the left side of the frame serves only as visual reference; the robot's motions were not extracted from the human video. All movements on the robot's side were separately scripted and executed in Isaac Sim with the physics engine enabled. This means the motions are not simple animation playback but are genuinely constrained by physics in simulation.

Physics Simulation Is Not Animation: Stability Is the Challenge

The developers admitted that keeping the G1 stable while interacting with objects "took considerable effort." Contact and sliding in simulation are still imperfect, and some flaws are visible in the video. This precisely demonstrates the value of physics simulation—it exposes problems that pure animation would never encounter.

In humanoid manipulation tasks, contact stability has always been a core challenge. The robot needs to maintain balance during grasping, placing, and pouring motions while handling object sliding and collisions. Isaac Sim's physics engine makes these interactions closer to real-world complexity and also provides a foundation for subsequent synthetic data generation.

Contact stability in rigid-body physics simulation is typically handled by a contact solver. Mainstream physics engines (such as PhysX, which Isaac Sim uses under the hood) employ constraint-based iterative solving, computing normal forces and friction at contact points during each simulation step so that objects neither penetrate each other nor produce non-physical bouncing. When multiple contact points exist simultaneously, mass ratios are extreme, or time steps are too large, the solver may fail to converge sufficiently, manifesting as object jitter, slow sinking, or objects slipping out of grasp. Humanoid robots are especially sensitive: while the arms contact objects, the feet must also maintain support against the ground, forming a closed-loop constraint chain along which errors accumulate. Common mitigation methods include reducing simulation step size, increasing solver iterations, lowering object mass ratios, tuning contact stiffness and damping, and using impedance control at the joint actuation level to absorb residual errors. This also explains why scripted motions look smooth in animation but expose flaws once physics is enabled.

Script-Driven, Not Learned Policies

Currently, the experiment has not used learned policies or automatic trajectory generation. All robot motions are predefined by scripts. The developers explicitly stated that this is only the first step of exploration, and what they truly care about is whether such an environment can become a source of synthetic robot demonstration data, including providing data for robots with different body structures.

This approach is highly relevant to research directions in imitation learning and synthetic data generation. If physics simulation environments can reliably generate high-quality robot manipulation demonstrations, they could help alleviate the bottleneck of high-cost, limited-scale real robot data collection.

Imitation learning centers on enabling robots to learn policies from demonstration data, with common methods including behavior cloning and inverse reinforcement learning. Behavior cloning directly maps observations to actions—simple to implement but suffering from distribution shift: once the robot deviates from the demonstration trajectory, errors accumulate and amplify. Synthetic data generation attempts to produce demonstrations in bulk using simulation: collecting states, actions, and success determinations in simulation, then narrowing the sim-to-real gap through domain randomization or domain adaptation, thereby alleviating the high cost of real-world collection and the difficulty of collecting dangerous motions. Notably, scripted trajectories themselves can also serve as demonstration sources: as long as the script executes within a physics engine, it can record physics-constrained state-action sequences—it just lacks the natural diversity of human demonstrations. Sharing data across robots of different morphologies also involves the retargeting problem, i.e., mapping motions from one body structure to another, which typically relies on joint correspondence or kinematic retargeting algorithms.

Industry Significance and Open Questions

This experiment demonstrates the feasibility of digital twin environments for scripting humanoid robot tasks. However, there remain several questions to answer on the path from script-driven to automatically generated data: Is the fidelity of physical contact sufficient? Can demonstrations generated in simulation effectively transfer to real robots? How can robots of different morphologies share demonstration data within the same environment?

The developers are openly seeking input from researchers working on imitation learning or synthetic robot data. For practitioners focused on humanoid manipulation and simulation-based data generation, this experiment offers a worthwhile starting point.

One-line Summary

Developers recreated a bubble tea station in Isaac Sim, using scripts to drive a Unitree G1 through a tea-brewing process, exploring the feasibility of physics simulation as a source of synthetic demonstration data.

Paragraph Summary

Developers transformed a real self-service bubble tea station into a physically interactive 3D digital twin environment, having a Unitree G1 humanoid robot complete a five-step tea-brewing task via scripting in NVIDIA Isaac Sim. The robot's motions were not extracted from human reference videos but separately authored and executed in a physics-enabled simulation. The biggest challenge was stability during robot interaction—contact and sliding remain imperfect, which precisely demonstrates the value of physics simulation. No learned policies or automatic trajectory generation have been used yet; what the team truly cares about is whether this environment can reliably produce synthetic robot demonstration data, including serving robots with different body structures, and they are seeking input from the imitation learning and synthetic data communities.