tau-bench
An AI agent evaluation benchmark proposed by Stanford University and collaborating institutions, designed to assess the reliability of tool-augmented agents in real-world database interaction scenarios. It measures whether system states match expected outcomes and evaluates success rates across repeated task executions.
Timeline (last 90 days)
The UserProxyBench paper was written by Ashish Jain and Armaan Sandhu, and adds an evaluation layer on top of tau-bench to measure whether the simulated user completes the task.
tau-bench was proposed by the Sierra team, covers retail and airline customer service scenarios, and uses the final state of the database as the success criterion.
The tau-bench benchmark focuses on whether an agent reaches the expected database state and requires consistent success across multiple repeated attempts.
tau-bench is an agent evaluation benchmark proposed by Stanford University and other institutions, targeting the reliability assessment of tool-assisted agents in real database interaction scenarios.
tau-bench uses whether the system state after agent execution matches expectations as its criterion, rather than the output content of a single conversation.
All Facts (5)
The UserProxyBench paper was written by Ashish Jain and Armaan Sandhu, and adds an evaluation layer on top of tau-bench to measure whether the simulated user completes the task.
50%Unverifiedtau-bench was proposed by the Sierra team, covers retail and airline customer service scenarios, and uses the final state of the database as the success criterion.
50%UnverifiedThe tau-bench benchmark focuses on whether an agent reaches the expected database state and requires consistent success across multiple repeated attempts.
50%Unverifiedtau-bench is an agent evaluation benchmark proposed by Stanford University and other institutions, targeting the reliability assessment of tool-assisted agents in real database interaction scenarios.
50%Unverifiedtau-bench uses whether the system state after agent execution matches expectations as its criterion, rather than the output content of a single conversation.
50%