Unanimous Consistency Rule
An aggregation rule in AI evaluation requiring all repeated runs to produce identical results. It causes the error rate to rise as N increases, because it conjoins N opportunities for failure.
Timeline (last 90 days)
When unanimity is required, increasing the number of runs raises the error rate: at p=0.10, the error rate is 0.2710 for N=3 and rises to 0.6513 for N=10.
Unanimity (--require-unanimous), as a policy switch in evaluation frameworks, is in some cases worse than the complement of a coin flip, and its cost is rarely reported.
All Facts (2)
When unanimity is required, increasing the number of runs raises the error rate: at p=0.10, the error rate is 0.2710 for N=3 and rises to 0.6513 for N=10.
50%UnverifiedUnanimity (--require-unanimous), as a policy switch in evaluation frameworks, is in some cases worse than the complement of a coin flip, and its cost is rarely reported.
50%