TextVBench
A benchmark framework specifically designed to evaluate text rendering quality in video generation models. It formalizes text evaluation as a multi-objective, multi-instance spatiotemporal correspondence problem, supports T2V and Ref2V evaluation tracks, and enables separate assessment of spatial and temporal fidelity.
Timeline (last 90 days)
TextVBench formalizes visual text evaluation as a multi-objective, multi-instance spatiotemporal correspondence problem.
TextVBench filters candidate regions given known target text (target-conditioned candidate filtering) to reduce false positives.
TextVBench supports one-to-many mapping, allowing a single target text to correspond to multiple OCR recognition results.
TextVBench separately measures frame-level spatial fidelity and cross-frame temporal fidelity independently.
TextVBench uses a length-weighted harmonic score to synthesize multiple evaluation factors.
TextVBench provides extra-text diagnostics to expose failure modes where the model hallucinates text not present in the prompt.
Preserving existing text from a reference image (Ref2V) is harder than generating text from a prompt (T2V), with a gap of approximately ten or more points between the two tracks.
Attribute slice analysis shows that model scores drop significantly as the number of reference text regions simultaneously present in the frame increases.
The TextVBench benchmark is organized around controlled attributes such as text, layout, carrier, and motion, allowing aggregate scores to be traced back to specific capability bottlenecks.
All Facts (9)
TextVBench formalizes visual text evaluation as a multi-objective, multi-instance spatiotemporal correspondence problem.
50%UnverifiedTextVBench filters candidate regions given known target text (target-conditioned candidate filtering) to reduce false positives.
50%UnverifiedTextVBench supports one-to-many mapping, allowing a single target text to correspond to multiple OCR recognition results.
50%UnverifiedTextVBench separately measures frame-level spatial fidelity and cross-frame temporal fidelity independently.
50%UnverifiedTextVBench uses a length-weighted harmonic score to synthesize multiple evaluation factors.
50%UnverifiedTextVBench provides extra-text diagnostics to expose failure modes where the model hallucinates text not present in the prompt.
50%UnverifiedPreserving existing text from a reference image (Ref2V) is harder than generating text from a prompt (T2V), with a gap of approximately ten or more points between the two tracks.
50%UnverifiedAttribute slice analysis shows that model scores drop significantly as the number of reference text regions simultaneously present in the frame increases.
50%UnverifiedThe TextVBench benchmark is organized around controlled attributes such as text, layout, carrier, and motion, allowing aggregate scores to be traced back to specific capability bottlenecks.
50%