Pixel-UMI
A method for learning environment-aware whole-body manipulation policies from sensorized human demonstrations. It includes components such as environment-aware retargeting and a retargeting-error-aware mask, and is used for cross-embodiment imitation learning in robotics.
Timeline (last 90 days)
Pixel-UMI proposes a complete pipeline for learning environment-aware whole-body manipulation policies from sensorized human demonstrations.
Pixel-UMI achieved an average task progress of 87.0% across five challenging tasks.
Environment-Aware Retargeting (EAR) combines the reconstructed scene geometry information with the head and gripper trajectories in the demonstration to determine the robot pose and generate collision-aware joint targets and base targets.
Pixel-UMI uses DAgger to collect human correction data under the online policy, in order to compensate for the shortcomings of offline retargeting data in contact details.
Pixel-UMI was evaluated on five categories of tasks, covering articulated object manipulation, deformable object handling, reaching in confined spaces, whole-body obstacle avoidance, and contact-intensive dual-arm coordination.
All Facts (5)
Pixel-UMI proposes a complete pipeline for learning environment-aware whole-body manipulation policies from sensorized human demonstrations.
50%UnverifiedPixel-UMI achieved an average task progress of 87.0% across five challenging tasks.
50%UnverifiedEnvironment-Aware Retargeting (EAR) combines the reconstructed scene geometry information with the head and gripper trajectories in the demonstration to determine the robot pose and generate collision-aware joint targets and base targets.
50%UnverifiedPixel-UMI uses DAgger to collect human correction data under the online policy, in order to compensate for the shortcomings of offline retargeting data in contact details.
50%UnverifiedPixel-UMI was evaluated on five categories of tasks, covering articulated object manipulation, deformable object handling, reaching in confined spaces, whole-body obstacle avoidance, and contact-intensive dual-arm coordination.
50%