MAMHOI
A framework for generating human-object interaction (HOI) in 3D scenes. It adopts an affordance-mediated decomposition strategy that splits scene understanding and motion synthesis into two independently trainable modules, avoiding reliance on paired human-object-scene data.
Timeline (last 90 days)
MAMHOI proposes an "affordance-mediated decomposition" framework that splits scene understanding and motion synthesis into two modules that can be trained independently.
MAMHOI adopts a two-stage pipeline: in the first stage, a scene-conditioned model receives 3D environment input and predicts feasible interaction locations and manners, and in the second stage, an affordance-conditioned HOI model generates the corresponding human-object motion sequences.
MAMHOI's decomposition framework does not require any paired human-object-scene samples; the scene understanding module can learn solely from human-scene data, and the interaction dynamics module can learn solely from human-object data.
Experiments in complex indoor environments show that MAMHOI significantly reduces object-scene penetration while better preserving human-object interaction quality.
MAMHOI chooses affordance as the intermediate representation, leveraging its shareability across agents and tasks to separate "environment actability," which is unrelated to specific human motion, out of motion synthesis.
MAMHOI's "decompose first, then connect" strategy can be borrowed for directions such as robot manipulation, virtual human animation, and embodied intelligence that are constrained by insufficient paired data.
The generalization ability of the affordance representation itself and the problem of two-stage error accumulation remain directions that MAMHOI needs to observe in the future.
All Facts (7)
MAMHOI proposes an "affordance-mediated decomposition" framework that splits scene understanding and motion synthesis into two modules that can be trained independently.
50%UnverifiedMAMHOI adopts a two-stage pipeline: in the first stage, a scene-conditioned model receives 3D environment input and predicts feasible interaction locations and manners, and in the second stage, an affordance-conditioned HOI model generates the corresponding human-object motion sequences.
50%UnverifiedMAMHOI's decomposition framework does not require any paired human-object-scene samples; the scene understanding module can learn solely from human-scene data, and the interaction dynamics module can learn solely from human-object data.
50%UnverifiedExperiments in complex indoor environments show that MAMHOI significantly reduces object-scene penetration while better preserving human-object interaction quality.
50%UnverifiedMAMHOI chooses affordance as the intermediate representation, leveraging its shareability across agents and tasks to separate "environment actability," which is unrelated to specific human motion, out of motion synthesis.
50%UnverifiedMAMHOI's "decompose first, then connect" strategy can be borrowed for directions such as robot manipulation, virtual human animation, and embodied intelligence that are constrained by insufficient paired data.
50%UnverifiedThe generalization ability of the affordance representation itself and the problem of two-stage error accumulation remain directions that MAMHOI needs to observe in the future.
50%