Cost per Unit of Performance
A core quantitative metric in model selection decisions, defined as the multiple of additional cost required per one percentage point of accuracy gain. It is used to compare the cost-effectiveness of options such as adding an advisor, switching to a stronger model, or increasing inference effort.
Timeline (last 90 days)
Cost per unit of performance is usually defined as the cost multiplier required for each 1 percentage point improvement in accuracy; in this case, 2.1 times the cost buys a 1.7 point improvement, which converts to about 1.24 times cost per point.
The cost-per-unit-of-performance ratio depends heavily on the task's baseline accuracy: the lower the baseline, the greater the relative benefit of a small improvement; when the baseline is already very high, the same marginal cost may buy only fluctuations close to noise.
All Facts (2)
Cost per unit of performance is usually defined as the cost multiplier required for each 1 percentage point improvement in accuracy; in this case, 2.1 times the cost buys a 1.7 point improvement, which converts to about 1.24 times cost per point.
50%UnverifiedThe cost-per-unit-of-performance ratio depends heavily on the task's baseline accuracy: the lower the baseline, the greater the relative benefit of a small improvement; when the baseline is already very high, the same marginal cost may buy only fluctuations close to noise.
50%