D&A: Decoupling Description and Assessment to Turn VLMs into Zero-Shot Robotic Reward Models

Why Do VLMs Struggle with Task Progress Estimation?

Vision-language models (VLMs) show potential for zero-shot video task progress estimation in robot learning, but existing frameworks share a fatal flaw: confirmation bias. Due to implicit success assumptions or relative temporal ordering designs, VLMs tend to "hallucinate" progress in failed executions. A more direct problem is that simply prompting frontier models degrades them into binary success detectors, unable to capture non-linear subtle progress and failure modes.

Post-training methods can mitigate this through explicit training, but at a high cost: they require large amounts of expensive real robot data, and dense post-training must be redone for every base model swap. This is clearly not a scalable path.

Confirmation bias in cognitive psychology refers to the tendency to search for, interpret, and remember information that supports one's existing hypotheses while ignoring contradictory evidence. In the context of VLM task progress estimation, this bias manifests as: when observing a video, if the model implicitly assumes "the task is heading toward success," it tends to interpret ambiguous or failed actions as "approaching the goal." For example, if a robotic arm has already missed the object, the model may still report "grasping in progress, progress increasing." The deeper cause of this bias is that VLM pretraining corpora are dominated by successful demonstrations, and autoregressive generation tends to continue existing narratives. Simple prompting of frontier models often degrades them into binary success detectors—precisely the combined result of confirmation bias and prompt design: the model only needs to judge "success or not" without distinguishing specific failure modes, amplifying hallucination risk.

D&A's Core Idea: Describe First, Then Assess

Decoupling Progress Estimation into Two Stages

D&A (Describe & Assess) decomposes progress estimation into two explicit stages. The first stage is objective, task-based textual description of scene dynamics—the model first observes the video and describes what happened in words, rather than directly assigning scores. The second stage is assessment: using explicit rubrics to judge the description.

The key to this decoupling is that the model no longer relies on the video's internal timeline for relative ranking. Traditional methods ask the model to compare "is this frame closer to success than the previous one," which easily falls into local hallucinations; D&A instead forces the model to first synthesize a text-based absolute success definition before scoring. This definition is an external, global criterion against which assessment is anchored.

Explicit rubrics draw on the idea of scoring rubrics from educational assessment and human annotation: decomposing the abstract goal of "task success" into several independently judgeable dimensions or levels—for example, "whether the object was correctly grasped," "whether the object reached the target position," "whether tipping or spilling occurred during the process"—with clear criteria for each dimension. Compared to having the model directly output a continuous progress score, rubrics transform assessment into several discrete, verifiable judgment items, reducing room for the model to improvise. In D&A, rubrics are externally specified by the task definition and constitute global criteria rather than internal temporal cues from the video, so the model must describe the facts and then check each item, rather than performing relative ranking based on "looks closer to success." This design also makes scores comparable across different videos and embodiments, providing interpretable grounds for fine-grained partial progress.

Using Explicit Rubrics to Counter Confirmation Bias

The root of confirmation bias is the model's tendency to "confirm" success along existing cues. D&A's assessment stage requires the model to judge each rubric item, forcing it to attend to evidence inconsistent with the success definition. For example, if the task requires "place the cup on the plate," the model must first describe "the robotic arm grasped the cup, but the cup tilted during movement and liquid spilled," then judge based on the rubrics that this is a partial failure rather than success.

This design calibrates zero-shot estimation, mitigates confirmation bias, and provides fine-grained partial progress indicators—accurately penalizing failed executions while correctly interpolating intermediate states of successful runs.

Empirical Performance: Pure Prompting Matches Post-Trained Models

The paper's empirical results show that D&A generalizes across multiple tasks, embodiments, and benchmarks. As a purely prompt-based method, it matches or even surpasses post-trained reward models. This means usable reward signals can be obtained without collecting real robot data or training separately for each base model.

Moreover, D&A's reward estimates can support efficient downstream reinforcement learning. Reward signal quality directly affects RL efficiency, and D&A's fine-grained progress indicators carry more information than binary success detectors, aiding policy optimization.

Post-training refers to additional training of a base model on task-specific data after pretraining, commonly including supervised fine-tuning, preference optimization, or reward modeling. In robot learning, post-trained reward models typically require large amounts of real robot trajectories with progress annotations—costly and time-consuming; more critically, every base model swap requires re-collecting适配 data and retraining, making reuse difficult. A reward model is a model that provides scalar reward signals for reinforcement learning, whose quality directly determines the direction and efficiency of policy optimization. As a pure prompting method, D&A does not modify base model parameters, thus avoiding data collection and per-model post-training overhead; its fine-grained progress indicators contain more information than binary rewards, helping RL obtain more precise gradient signals on partial successes and failure modes.

Implications for Robot Learning

The lesson of D&A is that VLM capabilities need not be unlocked through expensive post-training. The problem often lies in prompt and assessment framework design—having the model describe objectively first, then judge against external criteria, is more reliable than having it directly predict progress. This offers a low-cost, generalizable path for zero-shot robotic reward modeling, and reminds researchers that before pursuing larger models and more data, re-examining task decomposition and assessment protocols may yield greater returns.

One-line Summary

D&A uses a decoupled describe-then-assess prompting framework to mitigate VLM confirmation bias, matching post-trained reward models zero-shot.

Paragraph Summary

The article points out that VLMs commonly exhibit confirmation bias in robotic task progress estimation, hallucinating progress in failed executions, and simple prompting degrades them into binary success detectors; post-training can mitigate this but requires expensive data and per-model retraining. D&A decouples progress estimation into two stages: first an objective textual description of scene dynamics from the video, then item-by-item assessment of the description against explicit rubrics, anchoring evaluation in an external global success definition rather than the video's internal timeline. Empirically, this pure prompting method matches or surpasses post-trained reward models across multiple tasks, embodiments, and benchmarks, without real robot data or per-model post-training, and provides fine-grained partial progress indicators that support efficient downstream reinforcement learning.