Pedestrian Detection Under Domain Shift: A Confidence-Controlled XAI Interpretability Audit

Why Autonomous Driving "Explainability" Needs to Be Audited

Perception models in intelligent vehicles are increasingly required to be explainable — not just to say "there's a pedestrian here," but to articulate why. Yet a long-overlooked question remains: when driving scenarios undergo domain shift, do these explanations remain faithful?

Domain shift refers to a model operating on distributions outside its training data — for example, switching from one urban dataset to another collected under different conditions. For safety-critical perception tasks, if explanation methods break down under domain shift, safety judgments made on the basis of those explanations could be dangerously misleading. This arXiv study targets exactly that blind spot.

Explainable AI (XAI) is an umbrella term for techniques that make model decision-making interpretable to humans. In perception tasks, XAI methods typically output heatmaps or saliency maps that highlight the image regions the model "relied on" when making a decision. Faithfulness is the core metric for evaluating explanation quality: a faithful explanation should genuinely reflect the model's internal computational dependencies, not merely appear plausible. The most common way to assess faithfulness is the deletion test — masking the regions highlighted by an explanation and observing whether model confidence drops sharply. If it does, the explanation is deemed to have correctly identified the model's critical dependencies. This evaluation framework assumes that explanations and the detection process are independent and can be audited separately. The central challenge uncovered by this study, however, is that when detection strength itself influences deletion test results, that assumption no longer holds.

Methodology: How to Audit Explanations from a Fixed Detector

The researchers selected a fixed YOLOv8s pedestrian detector and conducted a cross-domain audit on two mainstream pedestrian datasets, PIE and JAAD. Fixing the detector was a deliberate choice — it concentrates the variable on explanation faithfulness itself, rather than differences in model capability.

The audit employed a rigorous statistical toolkit:

• ROI-based D-Deletion: measuring whether explanations genuinely point to the model's critical dependencies by removing highlighted regions and observing confidence changes.
• Frozen confidence terciles: stratifying detection confidence to prevent high- and low-confidence samples from confounding the analysis.
• Rank-based tests, bootstrap intervals, and Holm correction: ensuring statistically robust conclusions under multiple comparisons.

The core objective of this combination is to disentangle "explanation faithfulness" from "detection strength" — to see clearly the true relationship between the two.

D-RISE (Detection Randomized Input Sampling for Explanation) is a post-hoc explanation method designed specifically for object detection, extended from the classical image classification method RISE. Its basic principle is to randomly generate a large number of occlusion masks on the input image, statistically determine which regions, when masked, cause the greatest drop in detection confidence, and back out the image areas most influential to the current detection result — ultimately producing a continuous saliency heatmap. Because D-RISE does not rely on the model's internal structure, it is a black-box explanation method theoretically applicable across architectures. But precisely because its evaluation logic is tightly tied to detection confidence, the coupling problem revealed in this study arises — the statistical "quality" of an explanation becomes difficult to separate from the "strength" of the detection itself.

EigenCAM is a white-box gradient/activation map method that directly uses the principal components of convolutional feature maps to generate saliency maps, without perturbing the input. It thus serves as a "perturbation-free" baseline. Comparing these two method types helps researchers determine whether the coupling phenomenon stems from a specific method or from the evaluation paradigm itself.

Key Finding: Explanation Faithfulness Is Strongly Coupled with Detection Strength

The study's most critical finding is that deletion-based faithfulness is strongly coupled with the detection strength at the time the explanation is generated.

For D-RISE, the Spearman correlation between D-Deletion faithfulness and detection strength falls between 0.70 and 0.82. This means the more confident the detector, the more "faithful" the explanation appears.

This finding directly undermines a common practice — naive confidence-stratified comparisons are unreliable. Without first controlling for detection strength, comparing explanation quality across different confidence levels risks attributing to the explanation method what is in fact driven by detection strength.

Domain Dependence Persists Even After Controlling for Detection Strength

When the researchers re-audited within fixed f0 intervals to control for detection strength, they found that D-RISE faithfulness still exhibited domain dependence in the middle f0 range. Specifically, D-Deletion on the PIE dataset was higher than on JAAD, and this difference remained significant after Holm correction.

In other words, even after removing the confounding effect of detection strength, explanation faithfulness still varies across data domains. This is a warning to any practice that relies on validating explanation methods on a single domain.

Spearman correlation is a rank-based, nonparametric measure of association ranging from -1 to 1, robust to outliers and suitable for measuring monotonic relationships without assuming linearity. In this study's context, a Spearman correlation of 0.70–0.82 indicates a strong relationship — meaning detection confidence alone can roughly predict the ranking of faithfulness scores, severely obscuring the actual "working quality" of the explanation method. Holm correction (Holm-Bonferroni correction) is a multiple comparison correction method used to control the overall probability of Type I errors (false positives) when a researcher conducts multiple hypothesis tests simultaneously. It is more powerful than the simple Bonferroni correction. Results that remain significant after Holm correction carry stronger statistical credibility.

Not Just a D-RISE Problem

To verify whether this coupling is unique to D-RISE, the researchers introduced a perturbation-free EigenCAM baseline as a comparison.

The result: EigenCAM's overall faithfulness was lower than D-RISE's, but it also exhibited score coupling. This is an important finding — it indicates that the coupling between detection strength and faithfulness is not a flaw of any particular explanation method, but is more likely inherent to the deletion-based evaluation paradigm itself.

This is a signal to the entire XAI evaluation community: when we measure explanation quality by "removing key regions and seeing how much confidence drops," the evaluation result itself is contaminated by the model's detection strength.

Implications for Safety-Critical Perception

This study's conclusion is unambiguous: safety-critical perception tasks require confidence-controlled XAI audits.

In autonomous driving and similar scenarios, having an "explainable" detector is far from sufficient. If explanation faithfulness quietly drifts with detection confidence and data domain, then any safety argument built on those explanations is fundamentally compromised. The methodology proposed — first controlling for detection strength, then conducting rigorous cross-domain statistical comparisons — provides a more robust template for future explainability evaluation.

For the industry, this means that when deploying explanation tools, faithfulness conclusions established on a single laboratory dataset cannot be directly extrapolated to the variable conditions of real roads. The "trustworthiness" of explainability itself requires ongoing auditing.

Confidence-controlled XAI audits mean in practice: before comparing explanation quality across different data domains or scenarios, detection confidence (i.e., f0 scores) must first be controlled within the same interval, and only then should statistical comparisons be made. This is essentially a covariate control strategy, analogous to stratifying by age or disease severity before comparing treatment efficacy in medical research. The "frozen confidence terciles" method used in the study divides samples into three tiers by detection confidence and independently audits explanation faithfulness within each tier, making cross-domain comparisons statistically fairer. The practical implication is clear: when regulatory bodies or automakers validate explainability tools for on-board perception systems, this control step should be incorporated into standard testing protocols — rather than relying solely on overall average faithfulness scores from a single scenario.