Conceptrobust policy fidelity
Robust Policy Fidelity
An AI evaluation metric that requires a model to answer correctly across all variants within a test family—covering different qualifications, identities, and formatting—before counting the family as passed. It simultaneously measures sensitivity to qualification differences and robustness to irrelevant variations, providing a stronger signal of systematic bias than item-level accuracy.
Timeline (last 90 days)
Oct 12
The robust policy fidelity metric is measured by family-level pass rate, so a score of 50% means half of the families passed all checks, not that half of the individual answers were correct.
Unverified50%