CRMArena-Pro
A benchmark dataset designed to evaluate language model agents in customer relationship management scenarios. The Pro version introduces multi-document reasoning tasks requiring agents to cross-reference structured enterprise documents such as call records, price lists, and installation policies.
Timeline (last 90 days)
The study is based on 100 lead qualification tasks in CRMArena-Pro.
CRMArena-Pro is a benchmark dataset specifically designed to evaluate the performance of language model agents in customer relationship management scenarios, with the Pro version introducing more complex multi-document reasoning scenarios.
The sales representative claimed in every call that the delivery timeline was acceptable, and in 76 of those calls claimed the budget was acceptable, with such assertions in 31 of the tasks directly contradicting the price list and installation policy.
All Facts (3)
The study is based on 100 lead qualification tasks in CRMArena-Pro.
50%UnverifiedCRMArena-Pro is a benchmark dataset specifically designed to evaluate the performance of language model agents in customer relationship management scenarios, with the Pro version introducing more complex multi-document reasoning scenarios.
50%UnverifiedThe sales representative claimed in every call that the delivery timeline was acceptable, and in 76 of those calls claimed the budget was acceptable, with such assertions in 31 of the tasks directly contradicting the price list and installation policy.
50%