Saliency Maps Fail in Dense Breasts? Multi-Source Audit Reveals Explainability Fairness Risks

In medical imaging AI, saliency maps are often used to demonstrate that a model is actually focusing on lesion regions rather than relying on irrelevant features. However, a study of mammography classifiers raises a sharp question: Are these explanation methods still reliable in dense breasts? Using a chance-normalized, confounder-aware audit framework, the study is the first to systematically evaluate saliency map performance in dense breasts and reveals the significant impact of case mix on explanation fairness.

Research Background: The Challenge of Dense Breasts and the Need for Explanations

Dense breast tissue can mask tumors on mammography, increasing the risk of missed cancers. Therefore, when an AI model claims to "see cancer," the reliability of its explanation is especially critical in dense breasts. Yet few prior studies have specifically tested the localization ability of saliency maps in dense breasts. This study fills that gap and poses a key question: Do saliency maps perform worse at localizing cancer in dense breasts? If so, why?

The study used 1,345 malignant views from the CBIS-DDSM dataset to evaluate Grad-CAM, Integrated Gradients, and occlusion maps generated by three convolutional neural networks (CNNs). Evaluation metrics included whether the peak hit the lesion and a chance-normalized lesion-to-background saliency ratio (in bits, where +1 indicates doubled saliency).

Chance normalization is a method used to correct the baseline value of explanation metrics. In saliency map evaluation, even if the model has not truly localized the lesion, its saliency is not uniformly distributed and may produce spuriously high scores due to image structure or tissue density. Chance normalization estimates the expected value of the metric in the absence of localization ability (the "chance level") and subtracts or ratio-adjusts the raw metric by this baseline, yielding a value that more purely reflects the model's localization ability. For example, the lesion-to-background saliency ratio is measured in bits, where +1 indicates doubled saliency, and chance normalization ensures that the ratio approaches zero under random or irrelevant localization, avoiding overestimation of model performance due to intrinsic saliency differences in background tissue. This is especially important when comparing breasts of different densities, because dense tissue may naturally exhibit greater saliency fluctuations.

Key Findings: Significant Gaps and the Impact of Case Mix

The unadjusted gaps are large: Grad-CAM peaks hit the lesion in 44% of dense breasts versus 58% in non-dense breasts. However, case mix—primarily the higher proportion of calcifications in dense breasts—explained half or more of each gap. This means that part of the difference is not due to the model "discriminating" against dense breasts, but rather to different distributions of lesion types.

Nevertheless, occlusion maps retained a significant mean gap of -0.56 bits, indicating that even after adjusting for case mix, some explanation methods still exhibit systematic bias in dense breasts. In addition, control experiments point to problems with the classifier itself: the unadjusted gap exceeded that of a conspicuity map based solely on image saliency, and the model relied less on lesions in dense breasts.

Case mix refers to differences in the distribution of case characteristics (such as lesion type, size, and location) across groups (e.g., dense vs. non-dense breasts). In this study, the proportion of calcifications was higher in dense breasts, and calcifications may be easier or harder for certain saliency maps to localize, thereby affecting overall metrics. Case-mix adjustment typically uses stratified analysis, matching, or statistical models (such as regression) to control for these confounders and isolate the effect of density itself. For example, if hit rates are simply compared between two groups, the difference may partly stem from different lesion types rather than the model's sensitivity to density. Therefore, adjusting for case mix is a key step in auditing explanation fairness, as it more accurately reveals whether the model has systematic bias against dense breasts.

Grad-CAM is a gradient-based visualization method that computes the gradient of the target class score with respect to the final convolutional layer's feature maps and performs a weighted sum of the feature maps to generate a coarse localization heatmap. Its advantages are that it requires no modification to the network architecture and is applicable to many CNNs, but its resolution is low and it may fail to precisely localize small lesions. Integrated Gradients assigns attribution values to each pixel by integrating gradients along the path from a baseline to the input, providing finer pixel-level explanations, but it is computationally expensive and sensitive to the choice of baseline. Occlusion maps generate saliency by systematically occluding local regions of the image and observing changes in the output; they are intuitive but computationally intensive and may be affected by the occlusion patch size. These methods have different principles, and their performance differences in dense breasts may stem from differing sensitivity to texture and density variations.

Multi-Source Cross-Validation: Consensus and Divergence

According to analyses from Bilibili UP creators and YouTube channels, the study's core conclusion—that case mix explains a large portion of the gaps, but occlusion maps still show significant bias—has been widely recognized. The consensus is that explanation fairness audits must account for case mix and should report density gaps both before and after adjustment.

However, there is disagreement on the source of the bias. One side argues that the model's reduced reliance on lesions in dense breasts is the main cause; the other emphasizes that the instability of saliency maps themselves in high-density tissue may lead to localization failures. Insights from a single source note that conspicuity maps based solely on image saliency cannot fully explain the gaps, suggesting that the model's internal mechanisms may involve more complex interactions.

Practical Recommendations: Audit Framework and Reporting Standards

The study recommends the following standards when reporting the fairness of saliency maps:

• Report density gaps both before and after case-mix adjustment;
• Evaluate each classifier separately;
• Use saliency conspicuity maps and lesion-dependence controls as baselines;
• Prioritize chance-normalized metrics, such as the lesion-to-background saliency ratio (bits).

These recommendations provide actionable standards for explainability evaluation in medical imaging AI, especially emphasizing that in challenging scenarios such as dense breasts, simple saliency maps may be insufficient to demonstrate model reliability.

Conclusion: Explanation Fairness Requires More Rigorous Auditing

The study reveals that saliency maps do exhibit performance gaps in dense breasts, but the magnitude and significance of the gaps vary by method. Case mix is an important confounder, but not the whole story. Occlusion maps retain significant bias, suggesting that some explanation methods may systematically underestimate lesions in dense breasts. Therefore, confounder-aware auditing of explanation methods is essential before clinical deployment. Future research is needed to develop more robust explanation techniques to ensure that AI decisions in dense breasts are transparent and trustworthy.

One-line Summary

Study reveals gaps in saliency map cancer localization in dense breasts; case mix explains part but not all of the bias.

Paragraph Summary

This study evaluated the reliability of Grad-CAM, Integrated Gradients, and occlusion maps for localizing malignant lesions in dense-breast mammography. Using 1,345 malignant views from CBIS-DDSM, it found that Grad-CAM peak hit rates were 44% in dense breasts versus 58% in non-dense breasts, but case mix (e.g., more calcifications) explained at least half of the gap. Occlusion maps retained a significant mean gap of -0.56 bits after case-mix adjustment, indicating systematic bias. Control experiments showed that the unadjusted gap exceeded that of conspicuity-only control maps, and the model relied less on lesions in dense breasts, pointing to problems with the classifier itself. The study recommends reporting density gaps both before and after case-mix adjustment and using saliency controls and lesion-dependence controls for each classifier, emphasizing the need for confounder-aware auditing before clinical deployment.