Single-Step Accuracy
A measure of the proportion of steps or single model calls in which an agent correctly executes an action. It is one of the basic metrics for evaluating AI agent performance but cannot reflect the error accumulation effects in long tasks.
Timeline (last 90 days)
In long-horizon tasks, single-step accuracy can be regarded as an upper-bound estimate of the probability of each step being correct, while task success rate approximates the product of the per-step correctness probabilities, with the gap growing larger as the number of steps increases.
Focusing only on single-step accuracy masks systemic risk in long tasks.
All Facts (2)
In long-horizon tasks, single-step accuracy can be regarded as an upper-bound estimate of the probability of each step being correct, while task success rate approximates the product of the per-step correctness probabilities, with the gap growing larger as the number of steps increases.
50%UnverifiedFocusing only on single-step accuracy masks systemic risk in long tasks.
50%