VDAs
A method that uses video diffusion model priors to generate realistic, multi-view-consistent and controllable virtual humans from a single image. It replaces 3DMM with a learned expression space and decouples view bias.
Timeline (last 90 days)
VDAs (Video Diffusion Avatars) is a method that leverages video diffusion model priors to generate photorealistic, multi-view consistent and controllable avatars from a single image.
VDAs replace 3DMM with a learned expression space to break through the expressive ceiling of 3DMM.
Disentanglement can eliminate the view bias of learned expression encodings, enabling the encoding to serve as a driving signal for multi-view consistent avatars.
VDAs delegate control of camera viewpoint and upper-body pose uniformly to proxy mesh rendering, while expressions go through the learned encoding channel, so the three are independent of one another and can be adjusted separately.
VDAs adopts a camera-triplet training strategy, completing training on a single A100 in only 34 hours and relying solely on public multi-view data.
On the novel-view reenactment task, VDAs on the BecomingLit dataset reduces facial LPIPS by 29% and improves JOD by 66% compared with the strongest single-image 3D avatar method.
Static 3DGS reconstruction probe experiments confirm that VDAs' generation results perform best in multi-view consistency.
VDAs can produce physical effects that explicit renderers struggle to achieve, such as hair dynamics, shadows, reflections and refractions—implicit knowledge learned from video data.
All Facts (8)
VDAs (Video Diffusion Avatars) is a method that leverages video diffusion model priors to generate photorealistic, multi-view consistent and controllable avatars from a single image.
50%UnverifiedVDAs replace 3DMM with a learned expression space to break through the expressive ceiling of 3DMM.
50%UnverifiedDisentanglement can eliminate the view bias of learned expression encodings, enabling the encoding to serve as a driving signal for multi-view consistent avatars.
50%UnverifiedVDAs delegate control of camera viewpoint and upper-body pose uniformly to proxy mesh rendering, while expressions go through the learned encoding channel, so the three are independent of one another and can be adjusted separately.
50%UnverifiedVDAs adopts a camera-triplet training strategy, completing training on a single A100 in only 34 hours and relying solely on public multi-view data.
50%UnverifiedOn the novel-view reenactment task, VDAs on the BecomingLit dataset reduces facial LPIPS by 29% and improves JOD by 66% compared with the strongest single-image 3D avatar method.
50%UnverifiedStatic 3DGS reconstruction probe experiments confirm that VDAs' generation results perform best in multi-view consistency.
50%UnverifiedVDAs can produce physical effects that explicit renderers struggle to achieve, such as hair dynamics, shadows, reflections and refractions—implicit knowledge learned from video data.
50%