Does Federated Learning Sacrifice Interpretability? A Study on Grad-CAM Faithfulness in Deepfake Detection

The Privacy-Transparency Dilemma: The Trust Predicament of Federated Deepfake Detection

Deepfake detection has become a critical line of defense against the threat of synthetic media, but large-scale deployment requires cross-institutional collaborative training on sensitive data. Federated learning addresses privacy concerns by sharing only model updates rather than raw data, yet it introduces new concerns: do the gradient-based explanations (such as Grad-CAM) that analysts rely on become distorted as a result?

According to a recent systematic study, the relationship between federated training and explanation quality is not a simple "decline" but exhibits non-monotonicity—when data distributions are close to independent and identically distributed (IID) or moderately heterogeneous, federated model explanation faithfulness can match or even exceed the centralized baseline; however, under extreme heterogeneity, explanations collapse entirely, becoming spatially incoherent and causally misleading.

Study Design: Comparing Grad-CAM Faithfulness Between Centralized and Federated Training

The study targets an EfficientNet-B4 detector, constructs five federated clients on the FaceForensics++ dataset, and sets varying degrees of client data heterogeneity. The core evaluation metric is the faithfulness of Grad-CAM explanations—that is, whether the explanation maps truly reflect the model's decision basis rather than merely appearing plausible.

By comparing a centralized training baseline with federated-trained models, the authors attempt to answer a key question: does privacy protection come at the cost of interpretability? The experiments cover scenarios ranging from near-IID to extreme heterogeneity, providing fine-grained reference for practical deployment.

Grad-CAM and Explanation Faithfulness

Grad-CAM (Gradient-weighted Class Activation Mapping) is a gradient-based visual explanation method: it takes the gradient of the target class score with respect to the final convolutional feature map, applies global average pooling to obtain channel weights, then computes a weighted sum of the feature maps and upsamples the result to generate a heatmap of the same size as the input, highlighting the image regions the model deems most critical. Its "faithfulness" refers to whether the heatmap truly reflects the evidence on which the model bases its decision, rather than merely being visually plausible. Common evaluation methods include deletion/insertion curves (progressively occluding highlighted regions and observing how quickly prediction confidence drops), pointing game (comparing explanation regions with human-annotated forgery traces), and randomization sanity checks. Low faithfulness means the explanation may mislead analysts in judging the basis for detection.

Non-Monotonic Relationship: Threshold Effect in Federated Explanation Quality

The study finds that degradation of explanation quality is not a universal cost of federated training. Under near-IID and moderate heterogeneity conditions, federated models can even match or exceed the faithfulness of the centralized baseline. This means that when client data distributions are relatively consistent, federated learning can balance privacy and transparency.

However, when heterogeneity reaches extreme levels, explanation maps become spatially incoherent and, on average, causally misleading. This degradation is identified as a threshold effect, closely tied to the intensity of client heterogeneity. In other words, there exists a critical point beyond which explanations are no longer trustworthy.

Heterogeneity Threshold: When Explanations Begin to Fail

The authors emphasize that this degradation is not a gradual, universal decay but a threshold phenomenon bound to the degree of heterogeneity. This provides practitioners with an actionable early warning signal: by monitoring differences in client data distributions, one can predict when explanations may fail.

Measuring Data Heterogeneity and Non-IID

"Heterogeneity" in federated learning refers to the degree to which each client's local data distribution deviates from the global distribution. The ideal case is independent and identically distributed (IID), where each client's data comes from the same distribution; in reality, it is often Non-IID—for example, forged videos collected by different institutions differ significantly in generation methods, compression parameters, and face poses. Studies typically quantify heterogeneity intensity using Dirichlet distribution parameters, label distribution divergence (such as KL divergence), or inter-client feature distances. The higher the heterogeneity, the harder it is for the globally aggregated federated model to maintain consistent behavior across clients, and the more inconsistent the gradient directions—this is precisely the potential mechanism by which explanation maps may become distorted. Understanding how heterogeneity is measured helps grasp the practical meaning of the critical point in the "threshold effect."

Practical Guidelines: Boundaries for Safe Deployment and Monitoring Priorities

The study offers practical recommendations to help determine when privacy-preserving deepfake detection can be safely deployed and when its explanations are no longer trustworthy. Key points include: assessing client heterogeneity levels, validating explanation faithfulness before deployment, and establishing continuous monitoring mechanisms.

For scenarios requiring forensic review, the causal faithfulness of explanations is crucial. Analysts rely on tools like Grad-CAM to decide whether to trust flagged video clips; if explanations are misleading, this can lead to incorrect judgments. Therefore, in extremely heterogeneous environments, reliance on federated model explanation outputs should be exercised with caution.

FaceForensics++ and EfficientNet-B4

FaceForensics++ is a benchmark dataset in the deepfake detection field, containing forged videos generated by various face-swapping and expression-reenactment methods, and providing different compression levels (raw, lightly compressed, heavily compressed) to simulate real-world propagation conditions; it is commonly used to evaluate detector generalization. EfficientNet-B4 is a medium-scale backbone in the EfficientNet family, balancing depth, width, and resolution through compound scaling, achieving high accuracy on image classification tasks with fewer parameters, and is therefore often chosen as a detector backbone. The combination of the two makes the experiments both representative and reproducible; however, note that whether the conclusions apply to other architectures (such as Vision Transformers or temporal models) still requires additional validation.

Multi-Source Perspectives: The Universal Challenge of Federated Learning Interpretability

The study corroborates several recent independent analyses. According to discussions by Bilibili creators and YouTube tech channels, federated learning in domains such as medical imaging and financial risk control also faces interpretability challenges due to data heterogeneity. The consensus is that federated training itself does not necessarily harm explanation quality, but data distribution differences are the core risk factor.

Partial disagreement concerns the universality of the threshold effect—some argue that the threshold location may differ across model architectures and tasks and requires targeted validation. This study provides the first systematic evidence for the deepfake detection field, but cross-domain generalization still requires more empirical work.

Conclusion: Privacy and Transparency Can Coexist, but Heterogeneity Warrants Caution

Federated learning is not destined to sacrifice interpretability. When data distributions are controllable, it can simultaneously satisfy privacy protection and forensic transparency requirements. However, extreme heterogeneity can trigger explanation collapse, posing a risk of causal misdirection. Practitioners should establish heterogeneity monitoring and explanation validation processes to ensure federated deepfake detection is used within trustworthy boundaries.

The study's code and data splits have been made public, providing a foundation for reproducibility and encouraging more systematic evaluations of federated interpretability.

One-line Summary

Explanation faithfulness in federated deepfake detection is non-monotonic; extreme data heterogeneity triggers explanation collapse.

Paragraph Summary

This study systematically examines the faithfulness of Grad-CAM explanations in federated learning-trained deepfake detectors. Using EfficientNet-B4 and FaceForensics++, it constructs five federated clients with varying degrees of heterogeneity and compares them with a centralized baseline. Results show that the relationship between federated training and explanation quality is not a simple decline but non-monotonic: under near-IID and moderate heterogeneity, explanation faithfulness can match or even exceed centralized models; however, when client data heterogeneity reaches extreme levels, explanation maps become spatially incoherent and causally misleading, exhibiting a clear threshold effect. The authors propose deployment recommendations accordingly, including monitoring client heterogeneity, validating explanation faithfulness before deployment, and establishing continuous monitoring mechanisms, and emphasize caution in relying on federated model explanations under extreme heterogeneity in forensic scenarios.