D&A
A zero-shot task progress estimation framework that decouples the description and evaluation stages. A VLM first objectively describes the scene dynamics in a video, and then explicit scoring criteria are used to judge the description, thereby mitigating confirmation bias.
Timeline (last 90 days)
D&A (Describe & Assess) decouples progress estimation into two stages: the first stage is an objective, task-based textual description of scene dynamics; the second stage judges the description using explicit scoring criteria.
D&A forces the model to synthesize a text-based absolute definition of success before scoring; this definition is an external, global criterion against which the evaluation is anchored.
D&A's explicit scoring criteria are externally specified by the task definition, turning evaluation into several discrete, verifiable judgment items and reducing the room for the model to improvise.
The paper's empirical results show that D&A, as a purely prompt-based method, matches or even surpasses post-trained reward models across multiple tasks, embodiments, and benchmarks.
The fine-grained progress metrics produced by D&A are more informative than binary success detectors and can support efficient downstream reinforcement learning.
As a purely prompt-based method, D&A does not modify the base model's parameters, thereby avoiding the overhead of data collection and per-model post-training.
D&A's design makes scores comparable across different videos and different embodiments, providing an interpretable basis for fine-grained partial progress.
The insight from D&A is that VLM capabilities do not need expensive post-training to be unlocked; the problem often lies in the design of the prompting and evaluation framework.
All Facts (8)
D&A (Describe & Assess) decouples progress estimation into two stages: the first stage is an objective, task-based textual description of scene dynamics; the second stage judges the description using explicit scoring criteria.
50%UnverifiedD&A forces the model to synthesize a text-based absolute definition of success before scoring; this definition is an external, global criterion against which the evaluation is anchored.
50%UnverifiedD&A's explicit scoring criteria are externally specified by the task definition, turning evaluation into several discrete, verifiable judgment items and reducing the room for the model to improvise.
50%UnverifiedThe paper's empirical results show that D&A, as a purely prompt-based method, matches or even surpasses post-trained reward models across multiple tasks, embodiments, and benchmarks.
50%UnverifiedThe fine-grained progress metrics produced by D&A are more informative than binary success detectors and can support efficient downstream reinforcement learning.
50%UnverifiedAs a purely prompt-based method, D&A does not modify the base model's parameters, thereby avoiding the overhead of data collection and per-model post-training.
50%UnverifiedD&A's design makes scores comparable across different videos and different embodiments, providing an interpretable basis for fine-grained partial progress.
50%UnverifiedThe insight from D&A is that VLM capabilities do not need expensive post-training to be unlocked; the problem often lies in the design of the prompting and evaluation framework.
50%