Beyond Multi-Stage Pipelines: Label-Correlation Ensemble Model Improves Retinal Disease Classification

Retinal fundus images often carry multiple disease labels simultaneously, and public datasets exhibit heterogeneity and severe long-tail distributions. Clinically, triage workflows tend to adopt staged designs: first a screening gate for filtering, then a disease-group router to route to classifiers for specific disease groups. But is this hierarchical structure really worth its complexity? A new study provides a negative answer through controlled comparative experiments and proposes a better label-correlation ensemble approach.

Research Background and Experimental Setup

The study constructed a unified 39-target label space integrating three public datasets—RFMiD 1.0, RFMiD 2.0, and ODIR-5K—totaling 12,652 images. On this unified space, the researchers compared three architectures:

• Three-stage gated cascade: 8 models in sequence—a screening gate, a disease-group router, and group-specific classifiers—simulating the hierarchical design of clinical triage.
• Flat model: 39 independent sigmoid outputs predicting each label directly.
• Label-correlation model: Replacing the flat linear head with 39 learnable label queries and a two-layer transformer decoder to explicitly model dependencies between labels.

All models share the same Swin-V2-B backbone, identical preprocessing pipeline, and training protocol, ensuring that performance differences stem solely from architectural design.

Long-tail distribution refers to highly imbalanced sample counts across categories in a dataset, where a few head categories account for most samples while many tail categories have scarce samples. In medical imaging, rare disease labels naturally exhibit this distribution, leading to insufficient learning and low recall for tail categories. In this paper, the support across the 39 target labels varies significantly—a textbook case of the long-tail problem, which is why macro-F1 and performance on low-support categories warrant attention during evaluation.

Macro-AUC is the macro-averaged version of AUC (area under the ROC curve): AUC is computed separately for each category, then arithmetically averaged across all categories. Unlike micro-AUC (which pools predictions across all categories), macro-AUC treats each category equally and is therefore more sensitive to tail-category performance. In this paper, the gated cascade's macro-AUC was suppressed to 0.747 precisely because hard-zero outputs severely damaged the ranking ability for some categories.

Staged Design Offers No Consistent Advantage

Experimental results show that the three-stage gated cascade did not deliver the expected benefits. By tracing each missed disease instance to the stage responsible for the loss, the researchers found that the two coarse-grained gates (screening gate and router) discarded 34.2% of true instances, while the fine-grained classifiers lost only 6.7%. The gates' hard-zero outputs also suppressed the cascade model's macro-AUC to 0.747.

This finding challenges the intuition behind hierarchical designs in clinical triage: in data-driven models, the recall loss caused by coarse-grained filtering far exceeds the efficiency gains it brings.

Label-Correlation Modeling Delivers the Largest Single Gain

Replacing the flat linear head with the label-correlation model raised the detection rate from 0.621 to 0.682 (reaching 0.689 with test-time augmentation). This is the largest improvement among all single changes. The label-correlation model uses a transformer decoder to learn co-occurrence patterns between labels, effectively leveraging inter-disease associations.

Transformer decoder was originally designed for sequence-to-sequence generation tasks, modeling dependencies between sequence elements via multi-head self-attention. In this paper's label-correlation model, 39 learnable label queries serve as decoder inputs, each corresponding to one label, interacting through self-attention layers to explicitly learn label co-occurrence patterns (e.g., diabetic retinopathy and macular edema often co-occur). This design transforms label prediction from independent sigmoid classification into joint inference—the key to the performance gain.

Test-time augmentation (TTA) refers to applying multiple transformations (e.g., flips, rotations, multi-scale scaling) to input images during inference, predicting on each transformed version, and then ensembling the results. TTA does not change model parameters; it improves robustness and accuracy solely by increasing prediction diversity. This paper adopts four-view TTA, further raising the label-correlation model's detection rate from 0.682 to 0.689, and it played a role in the final ensemble configuration.

Ensemble Strategy and Final Configuration

Under four-view test-time augmentation, ensembling the flat model with the label-correlation model achieved the best configuration on six of eight metrics: exact match 0.694, micro-F1 0.678, macro-F1 0.456, macro-AUC 0.972.

The study also found that ensembling the gated cascade with continuous-output models is harmful, because the gates' hard-zero outputs are incompatible with continuous probabilities. Additionally, negative sampling by data source is necessary; otherwise, models exploit dataset shortcuts to produce spurious AUC of 1.000.

Negative sampling in this context refers to filtering or adjusting the sampling strategy for negative samples (i.e., images not containing a given label) during training. Because the three public datasets (RFMiD 1.0, RFMiD 2.0, ODIR-5K) differ in annotation rules and disease prevalence, models may exploit dataset source as a shortcut—for example, if a dataset rarely annotates a certain disease, the model learns that "images from this dataset do not exhibit that disease," producing spuriously high AUC. Negative sampling by data source ensures that negative samples from each source are appropriately balanced during training, forcing the model to learn genuine pathological features rather than dataset artifacts.

Comparison with Recent Research

On a defined matching subset of categories, the ensemble model matched or exceeded four recent studies on macro-F1 and micro-F1, with the most pronounced advantage on low-support pathological categories. This indicates that label-correlation modeling and sound ensemble strategies are particularly effective for multi-label classification under long-tail distributions.

Practical Implications

For multi-label medical image classification tasks, a simple flat model combined with label-correlation modeling and test-time augmentation may be more effective than a carefully designed multi-stage pipeline. Staged designs require careful evaluation of their recall loss, especially when coarse-grained gates may discard a large number of true positive instances. At the same time, data source bias must be addressed when training across datasets, using strategies such as source-wise negative sampling to avoid spurious performance.

One-line Summary

Study shows no advantage for staged gated cascades in retinal disease classification; label-correlation modeling and ensemble strategies are more effective.

Paragraph Summary

This study compared a three-stage gated cascade, a flat model, and a label-correlation model on a unified 39-label space. Results show that the cascade design simulating clinical triage brought no gains: coarse-grained gates discarded 34.2% of true instances, and hard-zero outputs suppressed macro-AUC to 0.747. In contrast, the model using a transformer decoder to capture label correlations raised the detection rate from 0.621 to 0.682—the largest single gain. Ultimately, ensembling the flat model with the label-correlation model under four-view TTA achieved the best results on six of eight metrics, with macro-AUC reaching 0.972, and outperformed four recent studies on low-support categories. The study emphasizes that in multi-label medical image classification, simple models with label-correlation modeling and sound ensembling may outperform complex pipelines, while source-wise negative sampling is needed to avoid spurious performance.