Majority Voting
An ensemble decision-making method that aggregates the predictions of multiple models and selects the answer that occurs most frequently as the final output, commonly used to improve accuracy on reasoning tasks.
Core Facts
Timeline (last 90 days)
Teacher intervention pilot studies show that teacher preferences differ across tokens, and that through majority voting and KL-divergence-based selection, a better teacher can be identified for each token.
Majority voting may reinforce a "stable yet wrong" consensus, because multiple views may be consistently wrong.
The pass@k metric does not penalize redundant or incorrect reasoning paths and is sensitive to the value of k, so in practice it is often combined with multidimensional metrics such as majority-vote accuracy, reasoning-chain step correctness, and human evaluation.
The premise for majority voting to yield gains is that voters make mistakes on uncorrelated things, that is, errors are approximately independent, and models from different vendors are the closest thing to independent failure samples without building one's own model.
At the same compute cost, an even N is better than its adjacent odd N: at p=0.10, the error rate is 0.0280 for N=3, 0.00370 for N=4, and 0.00856 for N=5.
When multiple LLMs share similar pre-training data or alignment biases, majority voting may fail to correct shared biases, which is why the study additionally introduced independent human blind-test validation.
Systematic errors shared by the majority of reviewers cannot be eliminated through majority voting; errors concentrated within a subgroup of a particular vendor or architecture can, however, be cancelled out in voting by selecting a combination of reviewers from different subgroups.
Statistically, majority voting can effectively reduce error rates only when errors are mutually independent; when errors are highly correlated, consensus instead reinforces mistakes
All Facts (8)
Statistically, majority voting can effectively reduce error rates only when errors are mutually independent; when errors are highly correlated, consensus instead reinforces mistakes
75%UnverifiedTeacher intervention pilot studies show that teacher preferences differ across tokens, and that through majority voting and KL-divergence-based selection, a better teacher can be identified for each token.
50%UnverifiedMajority voting may reinforce a "stable yet wrong" consensus, because multiple views may be consistently wrong.
50%UnverifiedThe pass@k metric does not penalize redundant or incorrect reasoning paths and is sensitive to the value of k, so in practice it is often combined with multidimensional metrics such as majority-vote accuracy, reasoning-chain step correctness, and human evaluation.
50%UnverifiedThe premise for majority voting to yield gains is that voters make mistakes on uncorrelated things, that is, errors are approximately independent, and models from different vendors are the closest thing to independent failure samples without building one's own model.
50%UnverifiedAt the same compute cost, an even N is better than its adjacent odd N: at p=0.10, the error rate is 0.0280 for N=3, 0.00370 for N=4, and 0.00856 for N=5.
50%UnverifiedWhen multiple LLMs share similar pre-training data or alignment biases, majority voting may fail to correct shared biases, which is why the study additionally introduced independent human blind-test validation.
50%UnverifiedSystematic errors shared by the majority of reviewers cannot be eliminated through majority voting; errors concentrated within a subgroup of a particular vendor or architecture can, however, be cancelled out in voting by selecting a combination of reviewers from different subgroups.
50%