VGC: Correcting Geometric Distortions in Generated Videos with Synthetic Paired Data

Generated videos often exhibit geometric inconsistencies under camera motion: structures distort when the viewpoint changes, parallax becomes incoherent, and smooth camera movements can degenerate into cuts. A new study from RSS attributes this failure to a data problem and proposes Video Geometry Corrector (VGC)—a post-processing video-to-video model trained on synthetic paired supervision that corrects geometric errors without modifying the source generator.

Why Geometric Consistency Is a Weak Spot in Generated Videos

The study points out that in web-scale video corpora, most visible motion comes from dynamic objects, while the cameras capturing them are often static or weakly moving. This means the supervision signal from which models can learn "viewpoint changes cause appearance changes" is very sparse.

When models are asked to generate videos with pronounced camera motion, they lack sufficient priors to maintain 3D structural consistency. Structures distort under viewpoint changes, parallax relationships break down, and camera movements collapse into jump cuts—all direct consequences of this data deficiency. The authors therefore argue that geometric inconsistency is essentially a "data problem," not merely a model capacity problem.

This judgment explains why simply scaling up generative models struggles to fundamentally cure geometric flaws—the training distribution lacks sufficient camera motion samples, so models naturally cannot learn the correct geometric transformation rules.

Parallax and 3D structural consistency: Parallax refers to the phenomenon where, during camera motion, objects at different depths displace at different speeds on the image plane. It is a key cue for human visual perception of 3D structure and a core metric for judging whether generated video geometry is reasonable. When generative models fail to correctly model parallax, foreground and background move inconsistently, causing objects to look like "paper cutouts" pasted on the background, or structures to distort under viewpoint changes. Geometric consistency requires that every frame in a video sequence corresponds to the projection of the same 3D scene under camera motion; the model must implicitly learn the relationship between scene depth and camera motion to satisfy this constraint.

How VGC Works: Post-Processing Rather Than Retraining

VGC's design philosophy is "post-hoc correction." It does not modify any source generator; instead, it layers a video-to-video transformation model on top of generated results, mapping geometrically distorted videos to geometrically consistent versions.

To train this corrector, the authors constructed the VG-Pairs synthetic paired supervision dataset. The specific process is: starting from real videos with diverse camera motions and pose annotations, each video clip is treated as a "geometrically consistent target," and a pretrained image-to-video inpainting model, combined with trajectory-adaptive keyframes, is used to synthesize the corresponding "distorted version." This yields paired (distorted, consistent) training samples.

During training, VGC learns to restore from distorted input to consistent targets. Optical flow is used as an auxiliary cue to help the model disentangle camera motion from object motion—this is key to geometric correction, because only by correctly distinguishing the two can viewpoint-induced deformations be corrected without destroying the objects' own motion.

Image-to-video inpainting models and trajectory-adaptive keyframes: Image-to-video inpainting models typically refer to pretrained models capable of generating or completing video sequences based on a single frame image and motion conditions. In constructing VG-Pairs, the authors use it to "reproduce" the visual content of real videos but introduce geometric distortions in a controlled manner. Trajectory-adaptive keyframes refer to dynamically selecting keyframes based on camera motion trajectories, so that the synthesized distorted versions have controllable viewpoint change patterns over time, thereby simulating common geometric errors in generative models. The key to this synthesis strategy is that the distorted versions are close to real videos in content and appearance but deviate from correct geometric transformation rules, thus providing high-quality paired supervision signals for VGC.

Optical flow and motion disentanglement: Optical flow describes the pixel correspondence displacement field between adjacent frames. In geometric correction, optical flow contains global flow caused by camera motion and local flow caused by objects' own motion. VGC uses optical flow as an auxiliary cue to separate the two: camera motion corresponds to consistent global transformations (e.g., flow field patterns caused by translation and rotation), while object motion manifests as local anomalies. Only by correctly disentangling the two can the model correct viewpoint-related deformations without destroying the objects' own dynamic behavior; otherwise, object motion might be misjudged as geometric error and "corrected" away, causing content distortion.

Experimental Performance and Scalability

On two types of benchmarks—pose-conditioned and prompt-driven—VGC improved the geometric consistency of multiple source generators. This means the method's correction capability is not tied to any specific generator and has a degree of generality.

The authors also demonstrate the framework's scalability: after scaling up the backbone network, VGC can extend toward more general video refinement. This suggests a pragmatic improvement path—rather than training stronger generative models from scratch, it is better to layer a scalable geometric correction module on top of existing generators.

Implications for the Generated Video Field

VGC's value goes beyond being a correction tool. It reveals a blind spot in current video generation research: camera motion bias in data distributions has long been overlooked, and this bias directly limits models' 3D understanding capabilities.

The idea of synthetic paired supervision is also noteworthy. It is difficult to obtain paired data of "the same scene, same content, different geometric quality" in the real world, but distortion synthesis can construct large-scale training signals at low cost. This data construction strategy could potentially be generalized to other video quality dimensions, such as temporal consistency and physical plausibility.

Of course, as a post-processing module, VGC's correction capability is limited by the degree of information retained in the source video. If the source generator has already severely broken down geometrically, the room for post-processing recovery is limited. Additionally, the distribution gap between synthetic distortions and real generation distortions remains an issue that the method needs to continuously validate.

One-line Summary

Research attributes geometric inconsistency in generated videos to data deficiencies and proposes a post-processing correction model, VGC.

Paragraph Summary

Generated videos often exhibit geometric inconsistencies such as structural distortion, incoherent parallax, and even jump cuts under camera motion. An RSS study points out that the root cause lies in sparse camera motion supervision signals in training data—web videos mostly feature static or weakly moving cameras capturing dynamic objects. To address this, the authors propose Video Geometry Corrector (VGC), a post-processing video-to-video model trained on synthetic paired supervision that corrects geometric errors without modifying the source generator. VGC constructs (distorted, consistent) paired samples via the VG-Pairs dataset and uses optical flow to help disentangle camera motion from object motion. Experiments show that VGC improves geometric consistency across multiple source generators, and scaling up the backbone enables extension toward general video refinement.