EM-DETR
A foundation model for medical lesion detection based on exemplar-driven grounding, built on the DETR architecture. Instead of a fixed-category classification head, it localizes and identifies lesions using reference examples, making it suitable for joint learning scenarios involving multi-domain heterogeneous label spaces.
Timeline (last 90 days)
The paper proposes a student-teacher training strategy for combining medical image datasets with different label spaces, in order to mitigate catastrophic forgetting and improve feature learning.
EM-DETR is a detection framework based on example-based grounding, in which the model locates and identifies targets using reference examples rather than relying solely on a fixed class classification head.
The research team adapted the EM-DETR framework to a multi-domain joint learning setting, enabling it to learn jointly across multiple domain-specific datasets and achieve cross-modality fine-grained, example-driven detection.
On four key findings in chest X-rays (CXR), the model's image-level AUC scores exceed the previous state of the art by 1 percentage point.
On CXR and mammography, the model achieved leading mAP50, 2-3 percentage points higher than EM-DETR trained on only a single dataset.
EM-DETR incorporates the idea of example-driven detection into the DETR (Detection Transformer) architecture, using the Transformer's query mechanism to enable interaction between examples and image features.
This work demonstrates a path to building scalable foundation models from heterogeneous data: handling differences in label spaces through the student-teacher strategy, and achieving cross-modality joint learning through the example-driven detection framework.
All Facts (7)
The paper proposes a student-teacher training strategy for combining medical image datasets with different label spaces, in order to mitigate catastrophic forgetting and improve feature learning.
50%UnverifiedEM-DETR is a detection framework based on example-based grounding, in which the model locates and identifies targets using reference examples rather than relying solely on a fixed class classification head.
50%UnverifiedThe research team adapted the EM-DETR framework to a multi-domain joint learning setting, enabling it to learn jointly across multiple domain-specific datasets and achieve cross-modality fine-grained, example-driven detection.
50%UnverifiedOn four key findings in chest X-rays (CXR), the model's image-level AUC scores exceed the previous state of the art by 1 percentage point.
50%UnverifiedOn CXR and mammography, the model achieved leading mAP50, 2-3 percentage points higher than EM-DETR trained on only a single dataset.
50%UnverifiedEM-DETR incorporates the idea of example-driven detection into the DETR (Detection Transformer) architecture, using the Transformer's query mechanism to enable interaction between examples and image features.
50%UnverifiedThis work demonstrates a path to building scalable foundation models from heterogeneous data: handling differences in label spaces through the student-teacher strategy, and achieving cross-modality joint learning through the example-driven detection framework.
50%