Optical Flow
A technique in computer vision for estimating pixel-level motion between adjacent frames, used to describe the displacement of pixels in a video. It is traditionally applicable only to motion estimation over short inter-frame distances.
Timeline (last 90 days)
Optical flow typically estimates only pixel-level motion between adjacent frames, whereas vid2vid-long's correspondences are patch-level semantic matches across long-distance frames.
Optical flow describes the apparent motion vectors of pixels between adjacent frames, and tends to fail at occlusion boundaries, textureless regions, and sudden exposure changes, so the transported points may land on the wrong surfaces.
Approaches to solving temporal coherence in AI video include introducing optical flow constraints, using temporal attention mechanism models (such as Runway Gen-3 and Kling), and post-processing frame stabilization tools (such as RIFE interpolation or Deflicker plugins).
FL2VA is driven by optical flow estimation to drive generation, focusing on the modeling and propagation of motion trajectories.
The concept of optical flow can be traced back to the classic variational method proposed by Horn and Schunck in 1981, as well as the contemporaneous Lucas-Kanade method.
光流估计(通过 RAFT、GMFlow 等算法)用于量化视频运动信息,光流均值接近零表示静态场景,方差过大意味着剧烈场景切换或抖动,这类数据不适合训练
数据筛选可使用光流估计量化运动幅度、CLIP评分评估文本匹配度、MUSIQ筛除低质量视频
光流能捕捉丰富细节但计算量大且对遮挡敏感,骨骼关键点更鲁棒易编辑但丢失肌肉收缩、衣物褶皱等细节
光流计算连续两帧之间每个像素的位移向量形成稠密运动场,经典算法包括Lucas-Kanade和Horn-Schunck,RAFT等深度学习方法提升了精度
All Facts (9)
Optical flow typically estimates only pixel-level motion between adjacent frames, whereas vid2vid-long's correspondences are patch-level semantic matches across long-distance frames.
50%UnverifiedOptical flow describes the apparent motion vectors of pixels between adjacent frames, and tends to fail at occlusion boundaries, textureless regions, and sudden exposure changes, so the transported points may land on the wrong surfaces.
50%UnverifiedApproaches to solving temporal coherence in AI video include introducing optical flow constraints, using temporal attention mechanism models (such as Runway Gen-3 and Kling), and post-processing frame stabilization tools (such as RIFE interpolation or Deflicker plugins).
50%UnverifiedFL2VA is driven by optical flow estimation to drive generation, focusing on the modeling and propagation of motion trajectories.
50%UnverifiedThe concept of optical flow can be traced back to the classic variational method proposed by Horn and Schunck in 1981, as well as the contemporaneous Lucas-Kanade method.
50%Unverified光流估计(通过 RAFT、GMFlow 等算法)用于量化视频运动信息,光流均值接近零表示静态场景,方差过大意味着剧烈场景切换或抖动,这类数据不适合训练
50%Unverified数据筛选可使用光流估计量化运动幅度、CLIP评分评估文本匹配度、MUSIQ筛除低质量视频
50%Unverified光流能捕捉丰富细节但计算量大且对遮挡敏感,骨骼关键点更鲁棒易编辑但丢失肌肉收缩、衣物褶皱等细节
50%Unverified光流计算连续两帧之间每个像素的位移向量形成稠密运动场,经典算法包括Lucas-Kanade和Horn-Schunck,RAFT等深度学习方法提升了精度
50%