Benchmark

UserProxyBench

A benchmark that adds a user fidelity evaluation layer to interactive Agent benchmarks, bringing into the measurement scope whether the simulated user model itself faithfully executes the scripted user scenario.

Timeline (last 90 days)

Oct 15

The UserProxyBench paper was written by Ashish Jain and Armaan Sandhu, and adds an evaluation layer on top of tau-bench to measure whether the simulated user completes the task.

Unverified50%

All Facts (1)

Unverified

The UserProxyBench paper was written by Ashish Jain and Armaan Sandhu, and adds an evaluation layer on top of tau-bench to measure whether the simulated user completes the task.

50%

Source Articles

Your Agent Benchmark's "User" Is a Second Model, and 24.4% of Its Successful Runs Broke the User's ScriptOct 15