AutomationBench-AA
A benchmark for evaluating the performance of AI on automation tasks.
Timeline (last 90 days)
The Ling-3.1-flash 1019 version improved its completion rate on AutomationBench-AA from 62% to 68%.
AutomationBench-AA focuses on end-to-end automated task completion rates, covering dimensions such as tool invocation, multi-step planning, and error recovery.
AutomationBench-AA's 68% completion rate indicates that models still have about one-third of failure space in automated tasks, and there is still a gap to reliable production-grade agents.
All Facts (3)
The Ling-3.1-flash 1019 version improved its completion rate on AutomationBench-AA from 62% to 68%.
50%UnverifiedAutomationBench-AA focuses on end-to-end automated task completion rates, covering dimensions such as tool invocation, multi-step planning, and error recovery.
50%UnverifiedAutomationBench-AA's 68% completion rate indicates that models still have about one-third of failure space in automated tasks, and there is still a gap to reliable production-grade agents.
50%