EgoExplorer
A 100-hour-scale visual-motor-tactile multimodal dataset collected with a wearable system. It aligns first-person video, whole-body motion reconstruction, finger joint motion and tactile observations, and provides step-by-step textual descriptions. It is intended for research on humanoid robot loco-manipulation and motion retargeting.
Timeline (last 90 days)
EgoExplorer is a 100-hour vision-motion-tactile observation dataset built with a wearable system, focusing on occupational activity scenarios, aligning four types of signals — egocentric video, reconstructed whole-body motion, finger joint motion, and tactile observations — and providing step-level text descriptions.
EgoExplorer employs the Calibrate-Complete-Constrain pipeline to reconstruct whole-body motion under sparse body sensing conditions, comprising three steps: calibrating body geometry and sensor positions, using temporal priors to complete unobserved joint motion, and ensuring measurement/temporal/geometric consistency through multi-layer constraints.
Tactile data collection usually relies on wearable sensing gloves, and the weight and number of the devices themselves will in turn affect the collector's natural movements, which is one of the practical reasons why EgoExplorer adopts sparse sensing and requires a calibration and constraint pipeline.
The paper presents preliminary research on multimodal generation and conditional control, as well as humanoid robot retargeting with whole-body control in simulation, to verify the dataset's downstream usability.
EgoExplorer's contributions include: placing vision, motion, touch, and finger details into the same wearable collection framework; using a calibration-plus-constraint method to demonstrate that sparse sensing can also yield relatively high-quality motion reconstruction; and directly linking the dataset with humanoid retargeting, shortening the path from human demonstration to robot policy.
EgoExplorer's collection bias toward occupational activities means the scene distribution has a specific character, and the 100-hour scale is medium-sized among multimodal datasets.
All Facts (6)
EgoExplorer is a 100-hour vision-motion-tactile observation dataset built with a wearable system, focusing on occupational activity scenarios, aligning four types of signals — egocentric video, reconstructed whole-body motion, finger joint motion, and tactile observations — and providing step-level text descriptions.
50%UnverifiedEgoExplorer employs the Calibrate-Complete-Constrain pipeline to reconstruct whole-body motion under sparse body sensing conditions, comprising three steps: calibrating body geometry and sensor positions, using temporal priors to complete unobserved joint motion, and ensuring measurement/temporal/geometric consistency through multi-layer constraints.
50%UnverifiedTactile data collection usually relies on wearable sensing gloves, and the weight and number of the devices themselves will in turn affect the collector's natural movements, which is one of the practical reasons why EgoExplorer adopts sparse sensing and requires a calibration and constraint pipeline.
50%UnverifiedThe paper presents preliminary research on multimodal generation and conditional control, as well as humanoid robot retargeting with whole-body control in simulation, to verify the dataset's downstream usability.
50%UnverifiedEgoExplorer's contributions include: placing vision, motion, touch, and finger details into the same wearable collection framework; using a calibration-plus-constraint method to demonstrate that sparse sensing can also yield relatively high-quality motion reconstruction; and directly linking the dataset with humanoid retargeting, shortening the path from human demonstration to robot policy.
50%UnverifiedEgoExplorer's collection bias toward occupational activities means the scene distribution has a specific character, and the 100-hour scale is medium-sized among multimodal datasets.
50%