Benchmark
MegaRCA-Mix
An evaluation dataset designed for root-cause attribution of long-horizon AI agent failures. It contains 50 manually annotated failure cases covering execution-intensive long-task scenarios, used to assess the performance of root-cause attribution methods on large-scale execution trajectories.
Timeline (last 90 days)
Sep 15
The research team constructed a new evaluation set, MegaRCA-Mix, containing 50 manually annotated failure cases covering long-horizon, execution-intensive tasks.
Unverified50%
Sep 15
On MegaRCA-Mix, Continual Search improved GPT-5.5's F1 score from 0.349 to 0.498, an increase of over 40%.
Unverified50%