TwinCheck
An inference-phase verification strategy for stateful tool-calling agents. It constructs counterfactual "negative twin" candidates and applies bidirectional ranking-consistency checks via paired validators to accurately identify and correct erroneous tool calls, while avoiding confusion between valid interventions and resampling noise.
Timeline (last 90 days)
TwinCheck is a verification strategy that runs at inference time and verifies whether to replace the agent's proposed action by constructing counterfactual candidate actions (negative twins) bound to the current trajectory.
TwinCheck only enters the replacement consideration process when the execution trajectory satisfies the evidence conditions bound to the local failure hypothesis of the trajectory.
TwinCheck replaces the agent's original proposal only when the negative twin simultaneously passes the structural check and the pairwise verifier prefers it under both candidate orderings.
TwinCheck introduces a precise replay-based pairwise evaluation method that keeps the agent's parsed responses and actions fixed until the first accepted replacement, in order to isolate the intervention effect from resampling noise.
The researchers selected 159 multi-turn BFCL V4 tasks with complete precise replay paired data for the main analysis.
The full TwinCheck strategy improved the task success rate of the GPT-5.6 Sol model from 45.3% to 58.5%, with a 95% task bootstrap confidence interval of [8.2, 18.8].
No regression from success to failure was observed in the experiments.
The paper "TwinCheck: Evidence-Grounded Negative-Twin Verification for Stateful Tool Agents" was published on arXiv.
All Facts (8)
TwinCheck is a verification strategy that runs at inference time and verifies whether to replace the agent's proposed action by constructing counterfactual candidate actions (negative twins) bound to the current trajectory.
50%UnverifiedTwinCheck only enters the replacement consideration process when the execution trajectory satisfies the evidence conditions bound to the local failure hypothesis of the trajectory.
50%UnverifiedTwinCheck replaces the agent's original proposal only when the negative twin simultaneously passes the structural check and the pairwise verifier prefers it under both candidate orderings.
50%UnverifiedTwinCheck introduces a precise replay-based pairwise evaluation method that keeps the agent's parsed responses and actions fixed until the first accepted replacement, in order to isolate the intervention effect from resampling noise.
50%UnverifiedThe researchers selected 159 multi-turn BFCL V4 tasks with complete precise replay paired data for the main analysis.
50%UnverifiedThe full TwinCheck strategy improved the task success rate of the GPT-5.6 Sol model from 45.3% to 58.5%, with a 95% task bootstrap confidence interval of [8.2, 18.8].
50%UnverifiedNo regression from success to failure was observed in the experiments.
50%UnverifiedThe paper "TwinCheck: Evidence-Grounded Negative-Twin Verification for Stateful Tool Agents" was published on arXiv.
50%