GraftOOD
GraftOOD is a reinforcement learning fine-tuning method for video generation models. It grafts out-of-distribution high-quality video examples into the online RL pipeline to address the motion collapse problem during reward optimization. The method employs a shared velocity regression objective to unify group-relative reward correction and sequence-level demonstration supervision signals.
Timeline (last 90 days)
GraftOOD research reveals that in video generation reward optimization, video dynamics are weakened when the model chases proxy rewards, a phenomenon named 'motion collapse'.
Even when video-level motion quality rewards are introduced, the motion collapse phenomenon still occurs.
GraftOOD's approach is to replace a portion of online-sampled results with out-of-distribution (OOD) video examples within selected prompt groups.
GraftOOD adopts a shared velocity-regression objective function, combining two types of supervision signals: group-relative reward correction and sequence-level example supervision.
The GraftOOD method does not require generating additional trajectories for examples, nor does it require introducing additional supervised loss coefficients to balance the two types of losses.
The research was validated on the Wan2.1-1.3B model.
Compared to the baseline method FlowAWR, GraftOOD increases Dynamic Degree from 4.86 to 44.41, an improvement of nearly 9 times.
GraftOOD causes HPSv3 to slightly decrease from 8.10 to 7.32, while VisionReward remains essentially unchanged.
Under the primary experimental setting, GraftOOD leads in nine out of ten metrics in a horizontal comparison among three example-augmentation methods.
Ablation experiments show that increasing example usage does not yield consistent gains, but instead presents a trade-off relationship between dynamics and reward.
2 more timeline events
All Facts (12)
GraftOOD research reveals that in video generation reward optimization, video dynamics are weakened when the model chases proxy rewards, a phenomenon named 'motion collapse'.
50%UnverifiedEven when video-level motion quality rewards are introduced, the motion collapse phenomenon still occurs.
50%UnverifiedGraftOOD's approach is to replace a portion of online-sampled results with out-of-distribution (OOD) video examples within selected prompt groups.
50%UnverifiedGraftOOD adopts a shared velocity-regression objective function, combining two types of supervision signals: group-relative reward correction and sequence-level example supervision.
50%UnverifiedThe GraftOOD method does not require generating additional trajectories for examples, nor does it require introducing additional supervised loss coefficients to balance the two types of losses.
50%UnverifiedThe research was validated on the Wan2.1-1.3B model.
50%UnverifiedCompared to the baseline method FlowAWR, GraftOOD increases Dynamic Degree from 4.86 to 44.41, an improvement of nearly 9 times.
50%UnverifiedGraftOOD causes HPSv3 to slightly decrease from 8.10 to 7.32, while VisionReward remains essentially unchanged.
50%UnverifiedUnder the primary experimental setting, GraftOOD leads in nine out of ten metrics in a horizontal comparison among three example-augmentation methods.
50%UnverifiedAblation experiments show that increasing example usage does not yield consistent gains, but instead presents a trade-off relationship between dynamics and reward.
50%UnverifiedThe GraftOOD paper points out that external example supervision should serve as a complement to scalar reward feedback, not a replacement.
50%UnverifiedThe paper emphasizes that 'higher measured dynamics' should not be simply equated with 'coherent, visually plausible motion'.
50%