Agentic Benchmark
An AI evaluation method that simulates complete workflows, requiring a model to autonomously decompose tasks, call tools, maintain intermediate state and deliver verifiable results, distinguishing it from traditional single-turn question-answering evaluation.
Timeline (last 90 days)
Agentic benchmarks are fundamentally different from traditional single-turn question-answering evaluations: the former simulate a complete workflow, requiring the model to autonomously decompose tasks, call tools, maintain multi-step intermediate state, and deliver verifiable results.
Absolute scores on conventional agentic benchmarks are generally low; even leading models score less than 20% on the legal work test, because the tasks themselves simulate work that would take human experts hours or even days to complete.
All Facts (2)
Agentic benchmarks are fundamentally different from traditional single-turn question-answering evaluations: the former simulate a complete workflow, requiring the model to autonomously decompose tasks, call tools, maintain multi-step intermediate state, and deliver verifiable results.
50%UnverifiedAbsolute scores on conventional agentic benchmarks are generally low; even leading models score less than 20% on the legal work test, because the tasks themselves simulate work that would take human experts hours or even days to complete.
50%