Proxy Metric
A proxy metric uses an observable substitute quantity to approximate a true objective that is difficult to measure directly. Proxy metrics carry the risk of systematic gaming or goal drift.
Timeline (last 90 days)
Downstream harm refers to the actual consequences of bias in real business pipelines (such as a hiring system rejecting certain types of names, or a credit model lowering the limits of a certain group); the symmetry score in the audit is only a proxy indicator related to it and cannot establish that the entire pipeline is fair to users.
When evaluating a metric, ask: if a number is optimized to an extreme value, does the real goal improve in tandem? If the answer is no, then it is at best a vanity metric, and using it to drive an agent will inevitably lead to goal drift.
A proxy metric refers to using another available and related quantity to approximately measure the target concept when true labels are lacking; treating a proxy score as proof of capability is a common misreading trap in fraud detection research.
Seven days of silence is still only a proxy metric, because customers who are completely frustrated and decide not to call again will also remain silent for seven days, making them indistinguishable from cases that were genuinely resolved.
Typing speed is a proxy metric for measuring the bandwidth of information conveyed to AI, and what it truly measures is expressive bandwidth.
The inherent flaw of proxy metrics is that once optimization targets the metric itself, its correlation with the true variable weakens.
All Facts (6)
Downstream harm refers to the actual consequences of bias in real business pipelines (such as a hiring system rejecting certain types of names, or a credit model lowering the limits of a certain group); the symmetry score in the audit is only a proxy indicator related to it and cannot establish that the entire pipeline is fair to users.
50%UnverifiedWhen evaluating a metric, ask: if a number is optimized to an extreme value, does the real goal improve in tandem? If the answer is no, then it is at best a vanity metric, and using it to drive an agent will inevitably lead to goal drift.
50%UnverifiedA proxy metric refers to using another available and related quantity to approximately measure the target concept when true labels are lacking; treating a proxy score as proof of capability is a common misreading trap in fraud detection research.
50%UnverifiedSeven days of silence is still only a proxy metric, because customers who are completely frustrated and decide not to call again will also remain silent for seven days, making them indistinguishable from cases that were genuinely resolved.
50%UnverifiedTyping speed is a proxy metric for measuring the bandwidth of information conveyed to AI, and what it truly measures is expressive bandwidth.
50%UnverifiedThe inherent flaw of proxy metrics is that once optimization targets the metric itself, its correlation with the true variable weakens.
50%