Can Agentic Systems Bridge the Radiology Gap? Testing FDA-Approved Tools

Radiology AI has long developed along two paths: specialized models perform well on narrow tasks but cover a limited set of clinical findings; general-purpose models can handle a broader range of tasks but are prone to hallucination and inaccurate measurements. Combining the two through agentic systems holds promise for comprehensive radiology reporting, but most published systems are limited to a single modality or body part. A recent study describes a radiology agent framework that standardizes how an orchestrator calls modular tools to generate structured reports, and uses FDA-approved radiology AI device tools to measure the actual capabilities of such systems.

The Dilemma of Two Paths and the Opportunity for Agentic Systems

Specialized models achieve high accuracy on specific tasks, but each model covers only a few clinical findings, making it hard to assemble a complete report. General-purpose models can handle multiple tasks but often exhibit hallucination and measurement bias, especially when precise numerical values are required. Agentic systems, by having an orchestrator call multiple specialized tools, can theoretically balance breadth and accuracy. However, existing published agentic systems often target only one modality or one body part and lack generality. The framework proposed in this study attempts to break this limitation by connecting the orchestrator and modular tools through standardized interfaces to generate structured reports.

Agent System refers to a software architecture in which a large model or rule engine acts as an "orchestrator" that dynamically calls external tools, modules, or sub-models to accomplish complex tasks based on task goals. Unlike an end-to-end single model, an agentic system separates "reasoning and planning" from "concrete execution": the orchestrator decides which tools to call and in what order, while the tools are responsible for producing high-precision outputs within narrow domains. In radiology scenarios, this means the orchestrator can, on demand, combine specialized models for measuring nodules, detecting hemorrhage, segmenting organs, and so on, thereby covering a broader range of clinical findings while preserving the accuracy of each tool. The core challenge lies in the reliability of the orchestration strategy—having all tools available but calling them improperly will still miss findings, which is precisely the constraint that the simulation experiments in this article attempt to quantify.

Simulation Experiments: Separating Tool Availability from Orchestration Strategy

The research team simulated the report generation process on a large multimodal reporting corpus. The orchestrator received two tool catalogs: one was a catalog of finding categories integrated from FDA-approved devices, where each category was available only on its approved modality; the other was an unrestricted catalog that could request any category. There were also two planning starting points: based on clinical indications or based on the target report. Tool outputs were simulated from reference reports, thereby separating tool availability from orchestration errors.

Key Findings: Recovery Rate and Reasons for Missing Findings

With the approved catalog and clinical indications alone, the orchestrator recovered less than one-third of clinically actionable findings. When using the unrestricted catalog, the recovery rate improved significantly; planning from the target report further improved it. An attribution analysis of each missed finding showed that most misses belonged to categories for which no approved device covered the study's modality, while another third were categories or tools that were provided but not called. On held-out reports, adding the most frequently missing tools closed most of the gap with the unrestricted catalog, whereas a smaller orchestrator using the unrestricted catalog lost all of the gains. This indicates that tool availability and effective orchestration are two complementary constraints for comprehensive AI-assisted radiology reporting.

FDA-approved radiology AI device tools refer to radiology AI software that has passed premarket review by the U.S. Food and Drug Administration (FDA), such as 510(k) or De Novo. Such devices are typically approved for specific modalities (e.g., CT, mammography) and specific clinical findings (e.g., pulmonary nodules, intracranial hemorrhage), and their intended use is strictly limited in the approval documents. Therefore, they constitute a "restricted tool catalog": each tool can only be legally called on its approved modality and task, and cross-modality or out-of-scope use is not permitted. This explains why many findings could not be covered in the experiments—not because it was technically infeasible, but because there was a lack of corresponding approved devices. This setup makes the simulation more closely reflect the regulatory constraints of real clinical deployment and also highlights the practical significance of expanding the coverage of approved tools.

Recovery rate is the core metric used in this article to measure the comprehensiveness of an agentic system, defined as the proportion of "clinically actionable findings" successfully included in the report generated by the orchestrator out of all actionable findings in the target report. It differs from traditional sensitivity or accuracy because it simultaneously penalizes two types of failure: there being no corresponding tool in the tool catalog at all (availability gap), and the tool existing but not being called by the orchestrator (orchestration gap). By attributing each missed finding to one of these two causes, the study can separate the contributions of "insufficient tools" and "improper orchestration." This metric design makes the conclusions more actionable: if a low recovery rate mainly stems from availability gaps, priority should be given to expanding the tool library; if it stems from orchestration gaps, the orchestrator should be optimized.

Implications for the Deployment of Radiology AI

This study reveals the insufficiency of current FDA-approved devices in terms of coverage: many clinically actionable findings lack corresponding tools, especially on specific modalities. At the same time, the capability of the orchestrator is also crucial—even with a complete set of tools, improper orchestration will still miss a large number of findings. Therefore, improving the comprehensiveness of radiology AI requires a two-pronged approach: expanding the coverage of approved tools and optimizing orchestration strategies. For institutions hoping to deploy agentic systems, priority should be given to evaluating the completeness of the tool catalog and tuning the orchestrator, rather than simply stacking models.

Limitations and Future Directions

This study is based on simulated tool outputs and does not account for error propagation from real tools; in actual deployment, tool errors may further reduce the recovery rate. In addition, although the corpus is multimodal, it does not cover all clinical scenarios. Future work could explore how to dynamically expand the tool library and how to design more robust orchestrators to cope with missing or erroneous tools. Nevertheless, this framework provides a reproducible benchmark for systematically evaluating radiology agents and clearly identifies tool availability and orchestration as the main current bottlenecks.

One-line Summary

A radiology agent simulation shows that tool availability and orchestration strategy are the two major bottlenecks for comprehensive reporting.

Paragraph Summary

This study proposes a radiology agent framework that standardizes how an orchestrator calls modular tools to generate structured reports, and evaluates it via simulation on a multimodal reporting corpus. The experiments set up two tool catalogs (an integrated catalog of FDA-approved devices and an unrestricted catalog) and two planning starting points (clinical indications and target reports) to separate the effects of tool availability and orchestration errors. Results show that with clinical indications alone and the approved catalog, the orchestrator recovered less than one-third of clinically actionable findings; the unrestricted catalog greatly improved the recovery rate, and planning from the target report further improved it. Attribution of misses showed that most were due to the lack of approved devices covering the modality, while about one-third were tools that existed but were not called. The conclusion is that tool availability and effective orchestration are complementary constraints, and improving the comprehensiveness of radiology AI requires a two-pronged approach.