GUIDER: Bridging the Path–Answer Gap in Clinical Guideline Reasoning

Clinical guidelines encode evidence-based recommendations as conditional decision flows, which are a natural source of process supervision signals for training reasoning models—but using them directly runs into two thorny obstacles. A new study proposes the GUIDER method, which jointly optimizes a decision-flow controller and a reasoning policy, significantly outperforming existing reinforcement learning baselines on multiple clinical benchmarks.

Two Major Challenges in Clinical Guideline Reasoning

The paper distills the core difficulties of process supervision in clinical reasoning into two concepts.

Path–answer gap: A guideline-consistent decision flow does not by itself determine the correct answer for a specific patient. In other words, following the correct diagnostic and treatment path does not guarantee that the final decision is right.

Path multiplicity: The same clinical decision can be supported by multiple different valid decision flows. If supervision relies on a single reference flow, the model is confined to one trajectory.

Existing training methods each solve part of the problem, but none handles both challenges simultaneously. Answer rewards supervise only the final decision; process rewards are typically computed against a single reference flow; and supervised fine-tuning on annotated flows can teach the model to generate guideline-compliant reasoning, yet it constrains the model to one trajectory per case and may even reduce answer accuracy.

GUIDER's Solution: Co-evolution of Controller and Policy

The core idea of GUIDER is to train two components simultaneously.

• Decision-flow controller: Learns from guideline-derived supervision to generate multiple plausible decision flows, addressing path multiplicity.
• Reasoning policy: Generates a decision flow and a final answer for each patient case.

The key design lies in how process rewards are computed. GUIDER evaluates the policy's process reward over multiple flows generated by the controller, thereby covering path multiplicity; at the same time, this reward is gated by answer correctness to bridge the path–answer gap. Joint training allows process supervision to evolve alongside the policy rather than remain fixed.

Process Reward Model (PRM) is the key background here: unlike outcome rewards that score only the final answer, a PRM scores the intermediate steps of reasoning step by step, alleviating the reward sparsity problem. Traditional PRMs are typically trained on human-annotated step-level labels or a single reference trajectory, and thus tend to misjudge in "multiple valid solutions" scenarios—marking another equally reasonable path as wrong. GUIDER computes process rewards over multiple flows sampled by the controller, which in essence extends the PRM's supervision signal to multiple trajectories and uses final-answer correctness as a gating weight, so that process rewards both cover diversity and stay aligned with the target answer.

Co-evolution and on-policy reinforcement learning: GUIDER's joint training belongs to the on-policy optimization paradigm—the policy continuously interacts with the environment (here, the distribution of decision flows generated by the controller), and process rewards are recomputed as the policy updates rather than fixed on a static dataset. This contrasts with offline process-supervised fine-tuning (SFT on annotated flows): the latter freezes the supervision signal before training begins, so once the policy deviates from the reference flow it receives no effective feedback. Co-evolution lets the controller act as a "dynamic problem setter" that continually generates new plausible paths, preventing the policy from overfitting to limited trajectories and explaining why process supervision can evolve together with the policy.

Experimental Results: Gains in Both Weighted Accuracy and Guideline Adherence

On the MedGUIDE benchmark, GUIDER with a Qwen3-14B backbone achieved a weighted accuracy of 82.5 and guideline adherence of 73.5, outperforming the strongest clinical RL baseline by 3.3 and 3.1 points respectively.

Cross-backbone and cross-domain generalization were also validated: on both Qwen3-14B and Gemma4-12B backbones, GUIDER also outperformed CRPO, MRPO, and FaithMed on the out-of-domain benchmarks AMEGA-LLM, RealMedQA, and CancerGUIDE.

Weighted accuracy and guideline adherence are the two core metrics of the MedGUIDE benchmark. Weighted accuracy typically computes a weighted average of answer correctness under case difficulty or class imbalance, preventing easy cases from dominating the overall score; guideline adherence measures how closely the model's generated reasoning steps match the decision flows in clinical guidelines, and can be judged manually or by rules/models. The two correspond respectively to "is the answer right" and "is the process compliant"—precisely the two sides of the path–answer gap. GUIDER improves both metrics simultaneously, indicating that it does not trade one for the other but rather alleviates the tension between them.

Why This Direction Matters

A long-standing contradiction in clinical reasoning models is that the more emphasis placed on guideline-compliant process, the more likely final answer accuracy is sacrificed. GUIDER's combination of "multi-flow sampling + answer gating" frees process supervision from the constraints of a single trajectory while keeping process rewards tied to correct answers. This approach also has reference value for other domains that require process supervision—such as multi-step tool calling and complex planning tasks.

The paper also points to a more general question: how should process rewards be designed when the reference process itself cannot uniquely determine the outcome? Clinical guidelines are merely a typical scenario of this problem, not an isolated case.

One-line Summary

GUIDER combines multi-flow process rewards with answer gating to jointly train a controller and policy, cracking the challenge of clinical guideline reasoning.

Paragraph Summary

Clinical guidelines encode evidence-based recommendations as conditional decision flows, a natural process supervision signal for training reasoning models, but two major obstacles exist: the path–answer gap (a compliant path does not guarantee a correct answer) and path multiplicity (the same decision can be supported by multiple valid flows). GUIDER jointly trains a decision-flow controller and a reasoning policy: the controller generates multiple plausible flows from guideline supervision to cover diversity, while the policy computes process rewards over these flows, gated by answer correctness to bridge the gap, with joint training allowing process supervision to evolve with the policy. On MedGUIDE, the Qwen3-14B backbone achieves 82.5 weighted accuracy and 73.5 guideline adherence, leading the strongest clinical RL baseline by 3.3 and 3.1 points respectively, and outperforms CRPO, MRPO, and FaithMed on multiple out-of-domain benchmarks.