Eulerian Motion Guidance: A New Solution for Long-Sequence Drift in Image Animation

The Persistent Problem in Long-Sequence Image Animation: Drift and Identity Collapse

Controllable image animation is one of the core tasks at the intersection of computer vision and generative AI. The basic goal is straightforward: given a static image (such as a portrait photo) and a driving signal (which could be the motion from another video, a skeleton keypoint sequence, or an optical flow field), generate a video sequence that preserves the original content while exhibiting the specified motion.

In recent years, methods like AnimateDiff, FOMM (First Order Motion Model), and LivePortrait have significantly advanced the field. FOMM (2019) was the foundational work in this space, pioneering the use of sparse keypoints and local affine transformations to describe motion, enabling cross-domain animation transfer without manual annotation. AnimateDiff (2023) brought diffusion models into video generation by inserting temporal attention modules into pretrained image diffusion models to achieve high-quality controllable animation. LivePortrait (2024) focused on portrait scenarios, adopting implicit keypoint representations along with efficient stitching and retargeting modules to strike a good balance between real-time performance and quality. What these methods share is a reliance on some form of motion representation as a conditioning signal, and the quality of that motion estimation directly caps the quality of the generated output. These approaches are widely used in portrait driving, landscape animation, and similar scenarios — yet one problem has long remained unsolved: the longer the animation sequence, the faster the visual quality collapses.

A paper forthcoming at ACM Multimedia 2026, titled Eulerian Motion Guidance: Robust Image Animation via Bidirectional Geometric Consistency, proposes a new supervision paradigm to address exactly this pain point. The authors shared the core ideas and experimental results on Reddit, generating significant community interest.

To appreciate the value of this work, we first need to understand the structural flaw in existing approaches.

Cumulative Error in Lagrangian Guidance

Most mainstream controllable animation methods estimate optical flow using the first frame as the reference — a strategy known as "Lagrangian guidance." This seems intuitive: all frames are aligned back to the starting point. But it harbors a fundamental flaw.

To understand why, it helps to know the engineering constraints of optical flow estimators. Optical flow is the 2D vector field describing pixel motion across an image sequence. Among classical algorithms, Lucas-Kanade (1981) uses brightness constancy and spatial smoothness constraints with iterative weighted least squares — efficient, but with clear limitations on large displacements. Horn-Schunck introduces global smoothness regularization, similarly constrained by small-displacement assumptions. In the deep learning era, FlowNet (2015) was the first end-to-end optical flow learner, while RAFT (Recurrent All-Pairs Field Transforms, 2020) pushed accuracy to new heights via a 4D correlation volume and GRU-based iterative refinement, dominating the Sintel and KITTI benchmarks for years. Yet even state-of-the-art methods like RAFT have a hidden but critical accuracy boundary: estimation quality is far better for small displacements (typically <20 pixels/frame) than for large ones. When training data distributions (FlyingChairs, FlyingThings3D) primarily cover small-to-medium displacements, estimation quality degrades significantly once real-world motion exceeds that distribution. When objects move too far between frames, estimators tend to fall into local minima and produce systematic bias.

As the animation sequence grows longer, the displacement between the current frame and the first frame continuously increases. Once this displacement exceeds the reliable operating range of the optical flow estimator, estimation quality drops sharply — and the result is drift, smearing, and identity collapse. In plain terms, the person gradually "becomes someone else," or facial structure slowly blurs and distorts.

This is not a failure of model capacity. It is an inevitable consequence of the supervision signal itself degrading as the sequence grows longer.

A New Reference Frame: Eulerian Motion Fields Between Adjacent Frames

The researchers' core insight is to shift the reference frame from the "first frame" to "adjacent frames" — using an Eulerian motion field for supervision. The naming borrows from the classic framework in fluid mechanics, a conceptual pair originating from two great mathematical physicists of the 18th century. The Lagrangian description tracks the complete motion history of each fluid particle — like attaching sensors to particles and moving with them. This naturally captures trajectories, but computational cost grows linearly with time. The Eulerian description instead observes fluid state at fixed spatial locations — like measuring flow speed at a fixed point in a river. Each observation depends only on local information at the current moment, with no cost accumulation over time. Mapped to video animation: the Lagrangian perspective tracks the cumulative displacement of a pixel from the first frame to the current one, offering global consistency but accumulating error along the trajectory; the Eulerian perspective only observes what happens between two adjacent frames — each step is estimated independently, and errors do not propagate. This precise migration of physical intuition forms the core design philosophy of the proposed method.

Making Every Step a "Short Hop"

Under the Eulerian framework, each training signal measures the motion between adjacent frames — essentially a "short hop." No matter how long the generated sequence becomes, displacement between adjacent frames remains small, reliably staying within the optical flow estimator's accurate operating range.

This yields a key property: supervision error at each step is strictly bounded and no longer amplifies with sequence length.

Compared to the exponentially growing errors characteristic of Lagrangian methods, Eulerian guidance breaks the error accumulation chain at its source. Rather than struggling to repair distorted long-range optical flow, the method ensures the supervision signal stays in a reliable region from the very start — a design decision that addresses the root cause.

Bidirectional Geometric Consistency: Filtering Incorrect Correspondences

Adjacent-frame supervision comes with its own challenge: as objects move, newly exposed (dis-occluded) regions continuously appear. Occlusion and dis-occlusion are among the most difficult geometric phenomena in video generation — when object motion exposes previously hidden regions, those pixels have no valid corresponding points in the previous frame. Forcing supervision on them only introduces random or incorrect correspondences. In the optical flow community, the Sintel dataset's official evaluation treats occluded regions as a separate sub-task precisely because occluded pixels lack valid references — any estimate is essentially unsupervised extrapolation. The KITTI dataset similarly annotates occlusion masks explicitly. Many methods still apply supervision to the full image, ignoring the sharp drop in signal quality for occluded regions.

The researchers address this through a forward–backward cycle check: for each pixel, they compute the forward flow $f_{t \to t+1}$, then the backward flow $f_{t+1 \to t}$, and verify whether the pixel can "return to its origin." If the endpoint error $|f_{t \to t+1}(p) + f_{t+1 \to t}(p + f_{t \to t+1}(p))| < \epsilon$ is not satisfied, that pixel is masked out from supervision and its gradient contribution is completely filtered. The computational cost of this check is minimal (requiring only two warp operations), yet it effectively identifies occlusions, fast-motion regions, and estimation failures — conceptually similar to confidence-based pseudo-label filtering in semi-supervised learning.

This mechanism ensures the model is never trained on incorrect correspondences, fundamentally preventing pollution from low-quality signals. It is also the origin of the "Bidirectional Geometric Consistency" in the paper's title.

Experimental Results: Quality and Efficiency Improve Together

Ultimately, a method's worth must be backed by data. The research team presents a compelling set of comparisons on 100-frame generation tasks.

Significantly Superior Generation Quality

On FVD (Fréchet Video Distance, lower is better) — the standard metric for video generation quality — Eulerian Motion Guidance achieves 76.18, outperforming the strongest baseline at 79.20. FVD was proposed by Unterthiner et al. in 2019 and involves three steps: first, a pretrained I3D (Inflated 3D ConvNet) network trained on Kinetics-400 extracts spatiotemporal feature vectors from video clips — I3D captures both spatial appearance and temporal dynamics by inflating 2D convolutions into 3D; then multivariate Gaussian distributions are fitted to the features of real and generated video sets respectively; finally, the Fréchet distance (i.e., Wasserstein-2 distance) between the two distributions is computed. Unlike FID, which measures single-frame image quality, FVD simultaneously captures both intra-frame visual quality and inter-frame temporal consistency, making it sensitive to drift and identity collapse in long sequences — an ideal metric for evaluating long-video generation stability. It's worth noting that FVD results are sensitive to sample size and choice of I3D feature layer, so experimental setup consistency must be ensured when comparing across papers. In long-sequence tasks, a meaningful FVD improvement typically corresponds to perceptible visual quality gains.

Even more telling is the user study on portrait animation: in human subjective comparisons, the proposed method achieved a 94.4% win rate. This near-overwhelming preference demonstrates that the stability improvements from Eulerian guidance are visually apparent — observers clearly preferred its maintenance of identity consistency and image clarity across long sequences.

Training Speed Also Improves

An additional important benefit is training efficiency. Since adjacent-frame optical flow can be computed in a single batched pass, overall training speed improves by approximately 2.7×.

This is particularly notable — in many studies, robustness gains come at the cost of higher computational overhead. Here, the more reliable supervision signal actually yields higher computational efficiency, because there is no need for complex multi-pass iterative estimation of long-range optical flow.

Methodological Implications

From a research perspective, this work embodies a "return to first principles" philosophy. Rather than stacking more complex network architectures or introducing additional regularization terms, the authors revisited the choice of reference frame for the supervision signal — and by adopting a framework better aligned with the reliable operating range of optical flow estimators, they avoided error accumulation at the source.

For researchers and engineers working on video generation, digital avatars, portrait driving, and related areas, two takeaways are worth internalizing:

• Error accumulation problems often stem from reference frame choices, not purely from limitations in model capacity;
• Geometric consistency checks are a low-cost, high-reward tool for filtering low-quality supervision signals.

It should be noted that these results currently come from a single source (the paper authors' own Reddit post) and await independent replication by the community across more datasets and scenarios. Interested readers can view videos and side-by-side comparisons on the project page, and refer to the arXiv paper for complete technical details.

Key Takeaways