RadOnc-Agent: Orchestrating a Full Radiation Oncology AI Workflow with Large Language Models

When AI Capabilities Are Trapped in Silos

Artificial intelligence has made significant strides across individual radiation oncology tasks — from target volume delineation and dose plan generation to treatment outcome prediction. Nearly every step in the process has a dedicated model. Yet an uncomfortable reality remains: these capabilities are fragmented, scattered across different clinical stages, software environments, and data modalities.

Radiation therapy is inherently a longitudinal, continuous care pathway — beginning with treatment decisions, moving through planning and delivery, and extending all the way to follow-up. The siloed state of these models stands in stark contrast to this coherent clinical workflow. In practice, clinicians must constantly switch between multiple systems and manually transfer patient context — a process that is both inefficient and error-prone.

RadOnc-Agent, introduced in a recent arXiv paper, is a systematic attempt to address exactly this problem. Rather than yet another single-task model, it is an agentic framework that uses a large language model (LLM) as the orchestration core, with the goal of connecting disparate radiation oncology AI capabilities into a coordinated pipeline.

Four Clinical Phases, 26 Callable Functions

The core design philosophy of RadOnc-Agent is to formalize the radiation therapy workflow into four clinical phases, exposing 26 callable functions through a conversational interface.

At the heart of the system is an LLM controller responsible for three key functions:

• Intent mapping: Translating clinicians' natural-language expressions into schema-constrained function calls, ensuring that call parameters are valid and structurally sound.
• Context retention: Preserving patient information and workflow state throughout the longitudinal workflow, so that subsequent steps can build upon prior results.
• Request routing: Dispatching specific requests to the appropriate specialist services.

The elegance of this architecture lies in positioning the LLM as an orchestrator rather than an executor. The model handles understanding and scheduling, while actual domain-specific computation is delegated to specialist modules. This leverages the LLM's strengths in language comprehension and task planning, while avoiding the risks of having a general-purpose model directly perform high-stakes medical calculations.

Agentic Framework represents an important paradigm shift in LLM application in recent years. Unlike traditional single-turn question answering, an agentic framework allows an LLM to interact with external systems through "function/tool calling": after receiving user instructions, the model autonomously decides which tools to call, in what order, and how to use intermediate results to advance subsequent steps. This design upgrades the LLM from a "language model that answers questions" to a "scheduling brain capable of executing multi-step tasks." In a clinical setting, this means clinicians can describe their needs in natural language, and the model automatically decomposes the task and invokes the appropriate specialist computation modules (such as dose calculation engines or image segmentation models), without manual context-switching across multiple systems. Schema constraints serve as the safety mechanism: by pre-defining the legal parameter format for each function, they enforce that function calls generated by the LLM conform to specifications, preventing invalid parameters caused by model "hallucinations" from triggering erroneous operations.

Three-Tier Evaluation: From Single Functions to Real Patient Cases

The paper's evaluation design is quite rigorous, structured across three progressive tiers that cover the full chain from atomic operations to complete clinical scenarios.

Single Function Execution

The research team used 2,600 single-function requests (7,800 total repeated executions) to test whether the system correctly selects the target function. Results showed that RadOnc-Agent selected the expected function in 98.79% of single-function executions — a figure that demonstrates highly reliable intent recognition and function routing at the foundational level.

Cross-Phase Synthetic Scenarios

Next, 200 pre-defined synthetic cross-phase scenarios spanning all four clinical phases (600 executions total) were evaluated. The system completed 96.50% of scripted cross-phase workflows, demonstrating that it can maintain high completion rates on multi-step, cross-phase task coordination.

Real Patient Workflows

Most compelling were 120 workflow instances built from 60 de-identified patient records (360 clean executions), covering two representative workflow types: "decision-to-planning" and "planning-to-adaptive." RadOnc-Agent completed 96.67% of real patient workflow executions — indicating that the framework is robust not only on idealized synthetic data but also in complex scenarios that approximate real clinical conditions.

Two Critical Design Choices Revealed by Ablation Studies

The most valuable part of the paper uses comparative ablation experiments to validate the indispensability of two architectural design choices.

The importance of longitudinal state retention: Removing longitudinal state caused cross-phase workflow completion to plummet from 96.50% to 84.00% — a drop of more than 12 percentage points that directly demonstrates the value of maintaining patient and process context in continuous workflows. An agent without memory simply cannot handle a task like radiation therapy, where phases are tightly coupled.

The safety significance of schema and identity verification: Disabling schema constraints and identity verification caused the erroneous backend dispatch rate to spike from 0% to 95.28% in replay/test evaluations. In other words, without this validation layer, the system would almost always route requests to the wrong place. In a clinical setting, such misdispatch could have serious consequences — underscoring that schema constraints are not merely an engineering detail, but a fundamental safety baseline.

Ablation Study is a standard method in machine learning research for validating the contribution of individual modules: components are "removed" one at a time and the resulting performance changes are observed, thereby quantifying each component's actual impact. This approach is more convincing than simply reporting overall performance — if removing a module causes a significant performance drop, the necessity of that design can be proven empirically rather than relying on intuition alone. RadOnc-Agent's ablation study targets the two most central architectural choices: longitudinal state management and schema/identity verification. The 12-percentage-point drop from the former reveals the fundamental gap between "stateful" and "stateless" agents on multi-phase tasks; the near-catastrophic 95% misdispatch rate from the latter clearly establishes what might seem like an engineering detail as a critical line of defense for medical safety.

Technical Feasibility — But Far from Clinical Readiness

The paper's conclusions are worded with admirable restraint, and this is worth emphasizing. The researchers explicitly state that these findings establish the technical feasibility of an LLM orchestration architecture for coordinating heterogeneous radiation oncology capabilities and information — but do not demonstrate clinical correctness, clinical utility, or prospective benefit.

This is a commendable example of scientific honesty. Workflow completion rates above 96% measure whether "the system can run through the process as expected" — not whether "the medical judgments it produces are correct or whether they can genuinely improve patient outcomes." From engineering feasibility to clinical value, there remains a vast gap that can only be bridged by rigorous prospective clinical validation.

For the broader medical AI field, RadOnc-Agent offers a paradigm worth referencing: use an LLM for top-level orchestration, specialist modules for bottom-level execution, and strict schema constraints with state management to ensure safety and coherence. This "orchestration-based agent" approach may be better suited to high-risk, process-intensive, context-heavy domains like healthcare than the pursuit of a single all-powerful model.

Conclusion

RadOnc-Agent reorganizes the lengthy and fragmented radiation therapy care pathway into a coordinated system unified under LLM-driven orchestration. It demonstrates high completion rates across single-function selection, cross-phase coordination, and real patient workflow execution, while ablation experiments validate the central roles of longitudinal state management and safety verification.

Yet running successfully in technical terms is only the starting point. Whether this framework can actually enter the clinic and improve patient treatment outcomes remains a question that subsequent rigorous clinical research must answer. The measured restraint of this paper offers a welcome rational reference point amid the often breathless enthusiasm surrounding medical AI.