How Differential Privacy Ensures Reliable Ranking Inference for Large Models

Introduction: When Ranking Requires Privacy Protection

Ranking based on human pairwise comparisons is increasingly common in large language model evaluation, but individual preference records often contain sensitive information. How to perform reliable statistical inference while protecting privacy has become an urgent problem to solve. A recent paper proposes a method for score and ranking inference under the Bradley-Terry model using edge differential privacy, offering a new approach to this challenge.

The Bradley-Terry model is a classic probabilistic model for handling pairwise comparison data, widely used in ranking and preference learning. The model assumes that each individual (or item) has a latent strength parameter, and when two individuals are compared, the probability that the stronger one wins follows a logistic function form: (P(i \text{ beats } j) = \frac{\theta_i}{\theta_i + \theta_j}), where (\theta_i) and (\theta_j) are positive parameters. Based on this model, individual strengths and rankings can be inferred through maximum likelihood estimation or Bayesian methods. In large language model evaluation, the Bradley-Terry model is commonly used to convert pairwise comparisons (such as human preference judgments) into interpretable scores, but traditional inference methods do not consider privacy protection, and directly using individual preference data may leak sensitive information.

Differential privacy is a rigorous privacy protection framework that ensures the influence of individual data is obscured by adding random noise to the algorithm's output. Edge differential privacy is a variant of differential privacy on graph data or relational data, particularly suitable for pairwise comparison scenarios. Under edge differential privacy, two adjacent datasets are considered to differ by only one edge (i.e., one comparison record), and the algorithm's output distribution should be nearly indistinguishable on these two datasets (controlled by the privacy budget (\epsilon)). This definition directly protects the privacy of each comparison record, because an attacker cannot infer from the output whether any single comparison exists. In ranking problems, edge differential privacy can prevent leaking specific preference relationships between individuals while allowing overall statistical inference.

Core Method: Objective Perturbation and Leave-One-Out Analysis

The paper develops a new leave-one-out analysis for the objective perturbation estimator, establishing finite-sample bounds for Euclidean error and entrywise error. For sufficiently dense comparison graphs, these error bounds can be extended to smaller privacy budgets, including cases where the privacy term dominates the statistical term. This analysis provides a theoretical foundation for ranking reliability under privacy protection.

Confidence Intervals and Bias Correction

Through uniform asymptotic expansion, the authors obtain asymptotically valid confidence intervals for individual scores and pairwise score differences. To improve finite-sample performance, they combine plug-in ridge bias correction with Gauss-Laplace calibration, simultaneously addressing regularization bias and privacy-induced randomness. In addition, they establish a bootstrap procedure for simultaneous ranking inference, providing asymptotically valid confidence sets for individual rankings and supporting Top-K membership testing.

Objective perturbation is a common technique for achieving privacy-preserving estimation in differential privacy. Its core idea is to add noise to the statistical objective function (such as the negative log-likelihood) before optimization, thereby obtaining a privacy-preserving estimator. Compared with methods that directly add noise to data or output, objective perturbation better preserves the statistical properties of the estimator and facilitates theoretical analysis. Under the Bradley-Terry model, objective perturbation involves adding noise generated by a differential privacy mechanism to the log-likelihood of pairwise comparisons, and then solving a regularized optimization problem. Leave-one-out analysis is a theoretical tool for evaluating estimator stability. By analyzing the impact of removing a single data point (or a single edge) on the estimator, finite-sample error bounds and asymptotic properties can be derived. The paper combines the two to establish rigorous theoretical guarantees for privacy-preserving ranking estimation.

Gauss-Laplace calibration is a calibration technique for differential privacy inference, designed to combine the advantages of the Gaussian mechanism and the Laplace mechanism to improve confidence interval coverage performance under finite samples. In objective perturbation estimation, the introduction of privacy noise increases the variance of the estimator, while plug-in ridge bias correction can mitigate the bias caused by regularization. Gauss-Laplace calibration adjusts the width of confidence intervals while accounting for both regularization bias and the randomness of privacy noise, thereby obtaining more accurate interval estimates. Specifically, the method uses Gaussian approximation and Laplace approximation to calibrate quantiles, making the actual coverage probability of confidence intervals closer to the nominal level under finite samples. This is particularly important for small-scale comparison graphs or cases with smaller privacy budgets.

Clever Savings on Privacy Budget

A key finding is that when comparison probabilities are public and known, all proposed inference procedures require no additional privacy budget. This means that part of the statistical inference can be completed without consuming privacy resources, significantly improving the practicality of the method.

Empirical Evaluation: Simulation and Creative Writing Ranking

The authors evaluate finite-sample performance through simulations and apply the method to creative writing ranking for large language models. The results show that the proposed method can provide reliable ranking inference while protecting individual preferences.

Significance and Outlook

This research tightly integrates differential privacy with ranking statistical inference, providing rigorous tools for model evaluation in privacy-sensitive scenarios. As large model evaluation increasingly relies on human feedback, such methods are expected to become part of standard practice.

Note: This article is based on the paper abstract. For specific technical details, please refer to the original paper.

One-line Summary

The paper proposes a ranking inference method for the Bradley-Terry model under edge differential privacy, achieving both privacy protection and reliable statistical inference.

Paragraph Summary

Aiming at the privacy issues of human preference data in large language model evaluation, this paper proposes a method for score and ranking inference under the Bradley-Terry model using edge differential privacy. The authors develop a leave-one-out analysis of the objective perturbation estimator and establish finite-sample bounds for Euclidean error and entrywise error, applicable to dense comparison graphs and small privacy budget settings. Through uniform asymptotic expansion, they obtain asymptotically valid confidence intervals for individual scores and pairwise score differences, and combine plug-in ridge bias correction with Gauss-Laplace calibration to improve finite-sample performance. In addition, they establish a bootstrap procedure for simultaneous ranking inference and Top-K membership testing. A key finding is that when comparison probabilities are public and known, inference requires no additional privacy budget. Simulation experiments and a creative writing ranking application verify the effectiveness of the method.