Dex-One2Many
A method for learning multi-fingered dexterous hand manipulation from a single human video. It guides reinforcement learning through serialized scene graphs and adopts a real-to-sim-to-real pipeline to achieve zero-shot real-robot transfer.
Timeline (last 90 days)
Dex-One2Many converts a single human video into serialized scene graphs, and uses these graphs to guide reinforcement learning.
Dex-One2Many adopts a real-to-sim-to-real pipeline: human videos are parsed into scene graphs, policies are trained in simulation, and finally zero-shot transferred to a real multi-fingered dexterous hand.
Dex-One2Many's training process is entirely completed in simulation, without using any real robot demonstration data.
Under the seen configurations, Dex-One2Many surpasses the baseline method by 6.5%.
Under unseen scenarios, Dex-One2Many's generalization ability widens the advantage gap to 71%.
The paper evaluates on five tool-use and manipulation tasks.
The scalability of the Dex-One2Many method remains to be verified, including questions such as whether scene graph construction depends on manual design and the degree of automation from video to graph.
All Facts (7)
Dex-One2Many converts a single human video into serialized scene graphs, and uses these graphs to guide reinforcement learning.
50%UnverifiedDex-One2Many adopts a real-to-sim-to-real pipeline: human videos are parsed into scene graphs, policies are trained in simulation, and finally zero-shot transferred to a real multi-fingered dexterous hand.
50%UnverifiedDex-One2Many's training process is entirely completed in simulation, without using any real robot demonstration data.
50%UnverifiedUnder the seen configurations, Dex-One2Many surpasses the baseline method by 6.5%.
50%UnverifiedUnder unseen scenarios, Dex-One2Many's generalization ability widens the advantage gap to 71%.
50%UnverifiedThe paper evaluates on five tool-use and manipulation tasks.
50%UnverifiedThe scalability of the Dex-One2Many method remains to be verified, including questions such as whether scene graph construction depends on manual design and the degree of automation from video to graph.
50%