[KongchangAI]
Model

SyncNet

An audio-visual synchronization detection model based on a two-stream convolutional network, used to determine whether facial movement and speech in a video are synchronized, thereby localizing the active speaker.

Timeline (last 90 days)

Nov 11

The VoxCeleb 1 dataset was built through an automated pipeline from the VGG Face dataset using a HOG-based face detector, SyncNet, and a VGG Face CNN, so bias in the face recognition domain propagates directly into the speaker verification dataset.

Unverified50%
Nov 10

Common metrics for evaluating lip-sync quality include LSE (Lip Sync Error) and SyncNet-based confidence scores.

Unverified50%
Sep 16

The author considers the lip-sync score only a coarse filter, noting that the 12-frame sample is too small and LLM scoring is noisy, and that a more reliable approach would use MediaPipe keypoint measurements or SyncNet scoring.

Unverified50%
Sep 16

SyncNet outputs a lip-sync score by comparing mouth visual features in video frames with audio MFCC features, and is widely used in academic lip-sync benchmarks; MediaPipe's FaceMesh module can track 468 facial keypoints per frame.

Unverified50%

All Facts (4)

Source Articles