VeriCUA
A platform-agnostic framework for verifying the task completion status of computer-use agents (CUAs). It reframes verification as a process in which the agent actively explores the environment, dynamically collects evidence and evaluates it, and consists of two stages: constructing scoring criteria and dual GUI/code checking.
Timeline (last 90 days)
VeriCUA is a platform-agnostic verification framework in which the same pipeline applies to both online and offline scenarios, covers different observation modalities, and aims to verify the task completion status of computer-use agents (CUAs).
VeriCUA redefines verification as a process of dynamically collecting evidence and making an assessment within a shared pipeline, where the verifier actively explores the environment rather than passively waiting for observation inputs.
VeriCUA's pipeline has two stages: first, it checks the available context and builds a task rubric; then, for requirements not yet resolved, it chooses GUI inspection or code inspection and synthesizes the findings of both under explicit evidence rules.
The quality of the rubric directly determines the verification ceiling, and omitted requirements cannot be recovered by subsequent exploration, so constructing the rubric in the first stage is the most critical step in the entire pipeline.
Test-time exploration draws on the idea of test-time compute, investing extra computation at inference time to improve result quality through multi-step interaction, trial and error, or search; in VeriCUA the verifier can actively perform actions such as navigation, clicking, and querying, at the cost of latency and operational risk.
According to the paper's abstract, VeriCUA shows higher consistency with reference labels across multiple experiments, outperforming the compared baseline methods; the improvement is most pronounced on difficult cases in offline evaluation, and it is better at rejecting false completion claims in online scenarios.
The paper attributes VeriCUA's performance gains to complementary evidence and test-time exploration, i.e., the verifier's ability to actively explore the environment at test time directly contributes to improved judgment accuracy.
All Facts (7)
VeriCUA is a platform-agnostic verification framework in which the same pipeline applies to both online and offline scenarios, covers different observation modalities, and aims to verify the task completion status of computer-use agents (CUAs).
50%UnverifiedVeriCUA redefines verification as a process of dynamically collecting evidence and making an assessment within a shared pipeline, where the verifier actively explores the environment rather than passively waiting for observation inputs.
50%UnverifiedVeriCUA's pipeline has two stages: first, it checks the available context and builds a task rubric; then, for requirements not yet resolved, it chooses GUI inspection or code inspection and synthesizes the findings of both under explicit evidence rules.
50%UnverifiedThe quality of the rubric directly determines the verification ceiling, and omitted requirements cannot be recovered by subsequent exploration, so constructing the rubric in the first stage is the most critical step in the entire pipeline.
50%UnverifiedTest-time exploration draws on the idea of test-time compute, investing extra computation at inference time to improve result quality through multi-step interaction, trial and error, or search; in VeriCUA the verifier can actively perform actions such as navigation, clicking, and querying, at the cost of latency and operational risk.
50%UnverifiedAccording to the paper's abstract, VeriCUA shows higher consistency with reference labels across multiple experiments, outperforming the compared baseline methods; the improvement is most pronounced on difficult cases in offline evaluation, and it is better at rejecting false completion claims in online scenarios.
50%UnverifiedThe paper attributes VeriCUA's performance gains to complementary evidence and test-time exploration, i.e., the verifier's ability to actively explore the environment at test time directly contributes to improved judgment accuracy.
50%