Cost per Correct Answer
An efficiency metric used in AI agent evaluation, calculated as total cost divided by the number of successfully completed tasks. Equivalent to the unit output cost concept in economics, it penalizes both high failure rates and high per-attempt costs.
Timeline (last 90 days)
In 7 of the 12 model × task combinations, high only changed the price while accuracy stayed the same, with the cost per correct answer rising by 1.5 to 3.4 times.
On the arithmetic control task, all models' accuracy stayed the same but the cost per correct answer rose by 1.65 to 2.5 times.
All Facts (2)
In 7 of the 12 model × task combinations, high only changed the price while accuracy stayed the same, with the cost per correct answer rising by 1.5 to 3.4 times.
50%UnverifiedOn the arithmetic control task, all models' accuracy stayed the same but the cost per correct answer rose by 1.65 to 2.5 times.
50%