Can General-Purpose Generative AI Replace Real Medical Imaging Data for Model Training?

General-purpose generative AI synthetic medical images aid training and auditing, but clinical value needs multidimensional evaluation.
This study systematically evaluated the general-purpose image generation model Nano Banana for synthesizing medical images across chest X-ray, fundus photography, CT, and carotid ultrasound. Models trained solely on synthetic data achieved AUROCs of 0.877, 0.963, and 0.711 for pneumonia, retinal detachment, and kidney stone detection—comparable to real-data models—but calibration and clinical net benefit were not always superior. Mixed training produced a vascular age index that stratified external subjects into subgroups with significantly different BMI and blood glucose. Editing-introduced patterns stress-tested shortcut learning, dropping true-class probability from 0.996 to 0.822. The study argues that the clinical value of synthetic data should be judged through calibration, decision curve analysis, and clinician review.
Introduction: The Data Bottleneck in Medical AI and the Potential of Generative AI
High-quality, well-annotated medical imaging datasets have long been a core bottleneck constraining the development of medical visual AI. Real medical data is not only expensive to acquire but also faces challenges such as privacy compliance, the need for specialized annotation expertise, and scarce samples for rare disease subgroups. In recent years, the rapid progress of general-purpose image generation and editing models has prompted researchers to ask: can synthetic medical images replace or supplement real data to train diagnostic models?
A multimodal retrospective study systematically evaluated this question, testing the ability of the general-purpose image generation model Nano Banana to synthesize medical images. The study covered four modalities—chest X-ray, fundus photography, CT, and carotid ultrasound—and validated across three dimensions: model training, subgroup augmentation, and model auditing.

Study Design: Four Modalities, Synthetic Training, and External Validation
The research team adopted a rigorous retrospective design. For three modalities—chest X-ray, fundus photography, and CT—models were trained exclusively on synthetic images, then externally validated on real-world data and compared with models trained on real data. The carotid ultrasound model was trained on a private Human Phenotype Project cohort (ages 40-75), testing the effects with and without synthetic data augmentation.
Clinicians rated the plausibility of synthetic images, and evaluation metrics included AUROC, AUPR, calibration, and decision curve analysis. This multidimensional evaluation framework itself deserves attention—it moves beyond the limitations of a single discriminative performance metric and incorporates calibration and clinical net benefit into consideration.
Key Findings: How Do Models Trained on Synthetic Data Perform?
Discriminative Performance Approaches That of Real-Data Models
Under pure synthetic-data training, models achieved AUROCs comparable to real-data-trained models on three tasks: pneumonia detection at 0.877 (95% CI 0.843-0.910, n=14863), retinal detachment at 0.963 (0.948-0.975, n=1149), and kidney stones at 0.711 (0.600-0.823, n=4122). This result indicates that synthetic images produced by general-purpose generative models already have considerable training value in terms of discriminative features.
However, the study also noted that synthetic-data models did not always show superior calibration or net benefit. This means that in real clinical decision-making scenarios, looking only at AUROC may overestimate the practical value of synthetic data.
AUROC (area under the receiver operating characteristic curve) is a common metric for measuring the discriminative ability of binary classification models, with 0.5 indicating random guessing and 1.0 indicating perfect discrimination. It reflects the model's ability to rank positive cases ahead of negative cases—that is, ranking ability—rather than the accuracy of probability estimates. Therefore, a high AUROC does not necessarily mean the model's output risk probabilities are trustworthy—which is precisely why the original article emphasizes calibration. AUPR (area under the precision-recall curve) better reflects a model's ability to identify the minority class than AUROC when classes are highly imbalanced; since positive samples are often scarce in medical settings, AUPR has complementary value.
Calibration describes how well a model's predicted probabilities match actual event frequencies. A well-calibrated model that predicts a 30% risk for 100 patients should see approximately 30 of them actually develop the event. Poor calibration leads to distorted threshold-based decisions and is commonly assessed using calibration curves and the Brier score.
Decision curve analysis (DCA) evaluates a model's practical value in clinical decision-making by calculating "net benefit." Net benefit combines the benefit of true positives with the harm of false positives and incorporates a threshold probability reflecting the clinician's willingness to intervene. Unlike AUROC, DCA directly answers whether using the model to assist decision-making yields more clinical benefit than treating all or treating none, and thus is closer to real-world application scenarios. The original article advocates incorporating calibration and DCA into the synthetic data evaluation framework precisely because they can reveal clinical risks masked by discriminative metrics.
Mixed Training Reveals Links Between Vascular Aging and Metabolic Health
In the carotid ultrasound task, models trained on a mixture of synthetic and real data produced an ordinal vascular age index that successfully stratified 633 external subjects into subgroups with significant differences in BMI and blood glucose levels (p<0.01). This finding directly links vascular aging to cardiometabolic health and also demonstrates the potential of synthetic data for subgroup representation augmentation—for populations underrepresented in real data, synthetic data can supplement representation.
Editing Capabilities Can Be Used for Model Auditing
The study also explored the use of synthetic editing in model stress testing. After editing fundus photographs to add a moderate grid-like pattern, the true-class probability for central serous chorioretinopathy dropped from 0.996 to 0.822. This suggests the model relies on shortcut features, and controlled editing offers a means to proactively uncover such vulnerabilities.
In addition, clinician-guided editing improved the plausibility score of low-quality CT images from 3.10/5 to 4.30/5, indicating that human-in-the-loop editing workflows can significantly improve synthetic data quality.
Shortcut learning refers to a model relying on spurious features in the data that are correlated with labels but not causal, rather than learning genuine pathological signals. For example, a model may "take a shortcut" to high accuracy because markers on images, device artifacts, or imaging conditions are statistically associated with disease labels, yet its performance collapses under distribution shift. Proactively introducing or removing specific patterns through controlled editing and observing changes in model outputs is a systematic stress-testing approach that reveals the sources of model vulnerability better than passively collecting failure cases.
In-Depth Analysis: Where Are the Boundaries of Synthetic Data's Value?
The most noteworthy aspect of this study is not whether synthetic data can train high-AUROC models—the answer is now fairly clear—but rather how clinical value is assessed. The study explicitly argues that the clinical value of synthetic data should be judged comprehensively through calibration, decision curve analysis, and clinician review, rather than relying solely on discriminative performance.
This position has practical significance. AUROC measures ranking ability, but clinical decision-making requires reliable probability estimates. A poorly calibrated model, even with a high AUROC, may give misleading risk assessments near the decision threshold. Decision curve analysis further incorporates net benefit, bringing evaluation closer to real-world application scenarios.
Another dimension worth considering is the role of synthetic data in model testing and auditing. By introducing specific patterns through controlled editing and observing changes in model outputs, researchers can proactively probe whether a model relies on non-causal shortcut features. This approach is more systematic than passively collecting failure cases and provides a new tool for safety evaluation of medical AI.
Of course, the study also exposes limitations: the calibration advantage of synthetic data is unstable, clinical plausibility still requires clinician involvement to improve, and the external generalizability of the private cohort needs further validation. Whether general-purpose generative models can cover sufficiently diverse real-world clinical variation remains an open question.
Conclusion and Outlook
This multimodal study offers a cautiously optimistic answer: synthetic medical images produced by general-purpose generative AI can serve as a supplement to real data, playing a role in three scenarios—model training, subgroup representation, and controlled testing. But they are not a simple substitute for real data, and their clinical value must be assessed within a multidimensional framework of calibration, decision benefit, and expert review.
For medical AI developers, this means synthetic data is worth adding to the toolbox but should not be treated as a shortcut to bypass data governance and clinical validation. Future directions may lie in more refined human-in-the-loop editing workflows and the establishment of quality standards for synthetic data tailored to specific clinical scenarios.
One-line Summary
General-purpose generative AI can synthesize medical images to aid training and auditing, but clinical value requires multidimensional evaluation.
Paragraph Summary
This study systematically evaluated the value of the general-purpose image generation model Nano Banana in synthesizing medical images across four modalities: chest X-ray, fundus photography, CT, and carotid ultrasound. Models trained solely on synthetic data achieved AUROCs of 0.877, 0.963, and 0.711 for pneumonia, retinal detachment, and kidney stone detection, respectively—comparable to real-data models; however, calibration and clinical net benefit were not always superior. A vascular age index produced by mixed training stratified external subjects into subgroups with significantly different BMI and blood glucose levels. Introducing patterns through editing stress-tested the model's shortcut learning, with true-class probability dropping from 0.996 to 0.822. The study argues that the clinical value of synthetic data should be judged comprehensively through calibration, decision curve analysis, and clinician review.
Related articles

Abnormal Error Rate Spike in Claude Mythos 5.2 and Fable 5.2
Claude Mythos 5.2 and Claude Fable 5.2 show an abnormal spike in request error rates, marked as investigating on the official status page. This article covers the incident overview, practical impact on developers, and recommended responses.

ReAct Runtime Context Trimming: Why Claude Code Acts at 70%
A deep dive into ReAct runtime context trimming: why Claude Code compresses at ~70% usage, why tool calls dominate, why messages must be dropped in pairs, and why codebases don't need RAG.

AI Agent vs LLM: How Cursor and Claude Code Actually Work
An in-depth analysis of the essential difference between AI Agents and LLMs: LLMs generate code suggestions, while Agents modify files and run tests via tool calls. Using a Spring Boot bug fix as an example, it breaks down the workflows, permission control, and verification mechanisms of Cursor and Claude Code.