MEgoVista
A trajectory-level optimization method for 3D hand pose reconstruction. By jointly optimizing across an entire recorded sequence and combining reliable visual observations with temporal and anatomical constraints, it addresses reconstruction when the hand is occluded by objects from a monocular viewpoint.
Timeline (last 90 days)
MEgoVista's hand tracking method has two steps: the first step uses a detector to distinguish the camera wearer's own hands from other people's hands, the second step tracks across frames to keep identities consistent, and then 3D motion reconstruction is performed.
The original poster proposed three ways to handle the case where a tracker undergoes an identity switch: discard the entire recording, cut out the affected segment, or keep it but add an uncertainty label.
MEgoVista's first-step detector is essentially performing hand attribution classification in egocentric scenes, with one extra identity dimension compared to generic hand detection.
MEgoVista's second step of cross-frame tracking draws on the detection-association paradigm in multi-object tracking, but with stronger constraints, requiring the same identity to be continuous in time.
MEgoVista combines reliable visual observations (frames in which fingers are clearly visible) with temporal continuity and anatomical constraints (such as fingers not being able to teleport and joints having physiological limits on their range of motion) to perform hand reconstruction.
MEgoVista-type methods demonstrate the feasibility of the path of not requiring humans to change their behavior but instead letting the reconstruction algorithm adapt to occlusion, and this idea is applicable to any robot learning task involving hand-object interaction.
The core idea of MEgoVista is to upgrade the hand reconstruction problem from single-frame inference to full-sequence optimization, optimizing hand motion at the scale of an entire recording.
All Facts (7)
MEgoVista's hand tracking method has two steps: the first step uses a detector to distinguish the camera wearer's own hands from other people's hands, the second step tracks across frames to keep identities consistent, and then 3D motion reconstruction is performed.
50%UnverifiedThe original poster proposed three ways to handle the case where a tracker undergoes an identity switch: discard the entire recording, cut out the affected segment, or keep it but add an uncertainty label.
50%UnverifiedMEgoVista's first-step detector is essentially performing hand attribution classification in egocentric scenes, with one extra identity dimension compared to generic hand detection.
50%UnverifiedMEgoVista's second step of cross-frame tracking draws on the detection-association paradigm in multi-object tracking, but with stronger constraints, requiring the same identity to be continuous in time.
50%UnverifiedMEgoVista combines reliable visual observations (frames in which fingers are clearly visible) with temporal continuity and anatomical constraints (such as fingers not being able to teleport and joints having physiological limits on their range of motion) to perform hand reconstruction.
50%UnverifiedMEgoVista-type methods demonstrate the feasibility of the path of not requiring humans to change their behavior but instead letting the reconstruction algorithm adapt to occlusion, and this idea is applicable to any robot learning task involving hand-object interaction.
50%UnverifiedThe core idea of MEgoVista is to upgrade the hand reconstruction problem from single-frame inference to full-sequence optimization, optimizing hand motion at the scale of an entire recording.
50%