Compounding Error
A concept originating from imitation learning theory, referring to the situation where a student policy deviates from the teacher policy with small probability at each step, causing subsequent inputs to fall outside the teacher's training distribution, which leads to the gradual accumulation of errors and collapse of the trajectory.
Core Facts
Timeline (last 90 days)
Anthropic's article warns that guardrails impose higher costs on Agents and may produce compounding errors, and recommends extensive testing in sandboxed, isolated environments in combination with guardrails.
If the per-step success rate of a long-horizon task is p, the overall success rate of executing N steps is approximately p^N, which explains the compounding error problem.
For long-horizon tasks, sim-to-real transfer error compounds with the compounding error, so extensive real-machine validation is often needed before deployment.
All Facts (4)
In autoregressive prediction, errors at each step accumulate into the next, causing long-range predictions to degrade rapidly.
75%UnverifiedAnthropic's article warns that guardrails impose higher costs on Agents and may produce compounding errors, and recommends extensive testing in sandboxed, isolated environments in combination with guardrails.
50%UnverifiedIf the per-step success rate of a long-horizon task is p, the overall success rate of executing N steps is approximately p^N, which explains the compounding error problem.
50%UnverifiedFor long-horizon tasks, sim-to-real transfer error compounds with the compounding error, so extensive real-machine validation is often needed before deployment.
50%