FedPCD: Breaking the Fairness Deadlock in Federated Learning with Priority Constraints

The "Averaging Trap" in Federated Learning

The core value of Federated Learning (FL) lies in training a shared global model from data distributed across many clients — without ever centralizing raw data. This paradigm is highly attractive in privacy-sensitive scenarios, but it carries a frequently overlooked side effect: optimizing for "average performance" often comes at the cost of fairness for certain clients.

When clients have statistically heterogeneous data distributions, the global model tends to favor clients with larger datasets or more mainstream distributions, while clients whose data characteristics deviate from the mainstream get "marginalized" — exhibiting noticeably worse local accuracy. This performance imbalance is one of the most challenging obstacles to deploying federated learning in real-world settings.

Statistical heterogeneity is one of the most pervasive challenges in real-world federated learning deployments. In practice, different clients' data often originates from vastly different user populations or collection environments — for example, patient demographics vary across hospitals, and mobile usage habits differ across regions — resulting in large discrepancies in label distributions, feature distributions, and data volumes across clients. This scenario is commonly referred to in the literature as the Non-IID (non-independent and identically distributed) data problem. Standard federated aggregation algorithms like FedAvg assume that client gradients can be directly averaged to estimate the global gradient. Under Non-IID conditions, however, gradients from different clients point in wildly divergent directions, and simple averaging causes "gradient divergence" — ultimately converging the global model to a skewed solution that works reasonably for the majority of clients but performs extremely poorly for a minority. Understanding this mechanism is fundamental to grasping the problem FedPCD sets out to solve.

Limitations of Existing Fairness Approaches

The research community has proposed a variety of multi-objective federated optimization methods targeting fairness, primarily through three approaches: client re-weighting, gradient correction, and common-descent optimization formulations.

These methods do improve the performance of disadvantaged clients to some degree, but they share two structural shortcomings. First, many methods alter the direction of global optimization in pursuit of fairness — essentially trading global performance for fairness and potentially degrading overall accuracy. Second, they tend to treat all client objectives symmetrically, lacking a clear distinction between the "primary global objective" and "client protection," causing the optimization goals to become entangled and difficult to control.

In other words, existing methods struggle to simultaneously preserve global performance and protect disadvantaged clients, frequently falling into a zero-sum tradeoff between the two.

Existing fairness-aware federated optimization methods can be further broken down to understand their limitations. Client re-weighting methods (e.g., q-FedAvg) assign higher weights to clients with higher losses to boost disadvantaged parties, but weight adjustments directly reshape the global loss function, causing the model to deviate from its original aggregation objective. Gradient correction methods (e.g., MGDA and its variants) model multi-client objectives as a multi-objective optimization problem and search for Pareto-optimal solutions, but the resulting "common descent direction" is not equivalent to the global aggregation direction, making losses in global accuracy nearly unavoidable. Common-descent methods theoretically guarantee simultaneous improvement across all client objectives, but tend toward conservative solutions when client objectives conflict severely, limiting both convergence speed and final accuracy. The shared problem across all these methods is that they merge "global optimality" and "fairness protection" into a single optimization objective, causing the two to interfere with each other in ways that are difficult to control independently.

FedPCD: Decoupling Global Objectives from Client Protection

FedPCD (Priority-Constrained Federated optimization) proposes a conceptual shift: priority constraints. Its core design explicitly separates the global optimization objective from the client protection mechanism.

Specifically, FedPCD preserves traditional federated aggregation as the primary objective — meaning the global model's optimization direction is never distorted by fairness demands. On top of this, the framework only permits the minimal corrections necessary to satisfy client performance constraints.

This design philosophy of "primary objective first, minimal constraint intervention" is elegant: rather than making zero-sum tradeoffs between global accuracy and fairness, it embeds client protection as a set of constraints within the optimization process, applying only the minimum directional correction when necessary. This approach both preserves the foundation of global performance and establishes a protection floor for disadvantaged clients.

Why "Minimal Correction" Is the Key

The problem with traditional methods is over-correction — in order to boost individual clients, they make substantial adjustments to the global gradient, with the side effect of harming overall convergence quality. FedPCD's constrained optimization formulation limits corrections to "just enough to satisfy the constraints," minimizing interference with the primary objective. This is the theoretical basis for its ability to simultaneously maintain high global accuracy while improving disadvantaged clients.

Experimental Results: Global Accuracy and Fairness in Tandem

Across multiple heterogeneous federated learning benchmarks, FedPCD delivers impressive results, demonstrating consistent advantages over state-of-the-art baselines:

• Highest global accuracy: Achieves the highest global accuracy across all configurations, confirming that prioritizing the primary aggregation objective is indeed an effective strategy.
• Leading performance for disadvantaged clients: In nearly all configurations, FedPCD leads on worst-10% client accuracy — a key fairness metric — demonstrating that its protection mechanism for disadvantaged clients is substantively effective.
• Competitive tradeoff performance: FedPCD maintains a competitive tradeoff between global accuracy and client performance variance, without exhibiting the typical "robbing Peter to pay Paul" problem.

These results collectively demonstrate that the priority-constraint design is not merely a theoretical ideal, but an advantage that can be reliably reproduced in real heterogeneous data environments.

Faster Convergence

Beyond the dual advantages in accuracy and fairness, FedPCD also exhibits an important practical characteristic: faster empirical convergence.

Experiments show that FedPCD can achieve performance comparable to or better than baselines in significantly fewer communication rounds. In federated learning, communication overhead is often the dominant deployment cost, since each round involves distributing the global model to clients and uploading local updates back to the server — both requiring substantial network transfer. Fewer communication rounds means lower bandwidth consumption and shorter training cycles, which is especially valuable in resource-constrained settings such as edge devices and mobile clients.

Communication rounds are a critical metric for evaluating the practical viability of federated learning systems. In each round of federated training, the server distributes the current global model to participating clients; each client performs several gradient update steps on its local data, then uploads local updates (gradients or model deltas) to the server for aggregation. This process faces strict constraints in mobile or healthcare IoT scenarios: limited network bandwidth, devices that may go offline at any moment, and user privacy policies that restrict upload frequency. Consequently, the fewer communication rounds required to reach a target accuracy, the higher the system's usability and economic efficiency. The intuition behind FedPCD's accelerated convergence is that by strictly limiting corrections to the minimum required by constraints, it avoids the oscillations and detours caused by over-correction in traditional fairness methods, making the optimization path more direct and efficient.

Implications for Real-World Federated Learning Deployment

FedPCD's significance extends beyond a new algorithm. It offers a framework for rethinking fairness in federated learning: rather than making vague weighted compromises between global objectives and fairness, decouple the two — keep the primary objective pure and translate fairness requirements into controllable constraints.

This "primary objective + minimal constraint correction" paradigm has direct practical relevance for real-world applications that must both guarantee overall service quality and avoid systematically disadvantaging any client — such as cross-hospital medical model collaboration or cross-institutional financial risk modeling. As federated learning moves from the lab to large-scale production environments, finding the right balance among performance, fairness, and communication efficiency is precisely the core question FedPCD aims to answer.