SPLIT-RL: Staged Training Tackles the Credit Attribution Problem in Vision-Language Reasoning

The Core Tension in Vision-Language Reasoning

When a vision-language (VL) model answers a question about an image, it actually needs to accomplish two fundamentally different things: accurately extracting relevant information from the image (visual reasoning, VR), and deriving the correct answer from that extracted information (language reasoning, LR). These two capabilities fail in completely different ways — the model might misread the image, or it might read the image correctly but reason to the wrong conclusion.

The problem is that the prevailing reinforcement learning training paradigm cannot distinguish between these two types of errors. Reinforcement learning with verifiable rewards (RLVR) typically uses a single chain-of-thought (CoT) paired with a final-answer reward to train both capabilities simultaneously. This approach has a structural flaw: it assigns the same sequence-level advantage to every token in the CoT, making it impossible to tell whether the error originated in the perception stage or the reasoning stage.

This paper on arXiv, titled SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages, proposes a solution to exactly this pain point.

RLVR is the dominant training paradigm for multimodal large models today, with GRPO (Group Relative Policy Optimization) as a representative method. GRPO samples multiple chains of thought for the same question, estimates the advantage function by comparing each trajectory's relative gap to the correct final answer within a group, and uses this to update the policy. The advantage of this approach is that it does not rely on human-annotated process rewards — only a verifiable final answer is needed. However, its underlying assumption is that "the entire chain of thought corresponds to a single signal source." For single-capability tasks (such as pure mathematical reasoning), this is reasonable. But once a task simultaneously involves two heterogeneous capabilities — perception and reasoning — the sequence-level advantage conflates the learning signals for both, leaving the model unable to determine which stage needs improvement.

Staged Training: Letting Advantage Signals Do Their Separate Jobs

The core idea behind SPLIT-RL is to split VR and LR training into two non-overlapping stages. The researchers' key insight is: if a group of rollouts (sampled trajectories) differs only along one capability dimension, the group-relative advantage can isolate that single capability and optimize it in a targeted way.

In other words, when training focuses on the visual reasoning stage, the variation among trajectories within a group primarily stems from the perception stage, so the advantage signal naturally points toward improving perceptual capability — and the same logic applies to the language reasoning stage. Each stage is optimized using its own phase-specific reward, rather than letting a generic final-answer reward blurrily cover all tokens.

The elegance of this design is that although training is split into two stages, the resulting policy is used at inference time in exactly the same way as a standard GRPO model — a single CoT call is all that's needed, with no added inference cost. SPLIT-RL thus operates entirely on the training side, imposing zero overhead at deployment.

Claim-Level Advantage: Breaking Visual Reasoning Down to the Atomic Level

Another key innovation in the paper is the Claim-Level Advantage (CLA-GRPO) mechanism.

Traditional approaches evaluate an entire visual reasoning trajectory as a single unit, but a single act of visual reasoning often involves multiple independent visual judgments. CLA-GRPO decomposes VR-stage rollouts into atomic visual claims and provides fine-grained advantage signals for each individual claim based on a visual-type group formation strategy.

This effectively refines the reward granularity from "is the entire reasoning passage correct?" down to "is each individual visual judgment correct?" For complex image tasks involving multiple perceptual elements, this fine-grained feedback more precisely tells the model exactly which step went wrong, making perceptual capability training significantly more efficient.

An "atomic visual claim" can be understood as the smallest verifiable assertion about a single attribute or relationship in an image — for example, "the object on the left in the image is red" or "there are three people in the image." Once a visual reasoning passage is decomposed into such atomic claims, each claim can be scored independently — using an external visual verifier or a language model cross-checking against the image — yielding a supervision signal far more granular than "is the whole passage correct?" Visual-type group formation further organizes claims by attribute category (color, count, spatial relationship, text, etc.), making within-group comparisons more meaningful and advantage estimation more robust. In essence, this design converts sparse outcome rewards into dense process supervision without requiring any human annotation at each step.

Experimental Results: Gains in Both Perception and Reasoning

The evaluation results validate the approach's effectiveness. Across the Qwen3-VL model family (from 2B to 30B-A3B) and InternVL3.5-8B, SPLIT-RL achieves an average accuracy improvement of 1.4 to 6.1 percentage points over standard GRPO. This improvement holds across models of different scales, indicating that the method generalizes well.

Even more convincing is the oracle-based diagnostic evaluation. Using this diagnostic approach to examine the two capabilities separately, the researchers found that GRPO — which relies solely on final-answer rewards — achieves almost no improvement in perception ability (perception unchanged). In other words, accuracy gains from GRPO are almost entirely attributable to the language reasoning side, while visual perception stagnates. SPLIT-RL, by contrast, delivers substantial improvements on both VR and LR dimensions.

This contrast is quite revealing: it exposes a hidden flaw in conventional end-to-end reinforcement learning. Because advantage signals cannot distinguish between capability sources, models tend to take the "easy path" and optimize the more-readily-improved reasoning stage, leaving the harder perception stage behind. SPLIT-RL's structured advantage isolation forces the model to strengthen both legs.

Oracle diagnostic evaluation is a common technique for decoupling capability analysis. At test time, researchers manually provide an "ideal visual description" to replace the model's own perception output, allowing the language reasoning module to answer under conditions free of perceptual error — thereby measuring the upper bound of reasoning capability in isolation. Conversely, the language reasoning strategy can be fixed while only the perception stage varies, measuring pure perceptual capability. By comparing accuracy changes under these two conditions, researchers can quantitatively attribute how much of a model's overall improvement comes from perception gains versus reasoning gains. This diagnostic approach reveals a latent issue with standard GRPO: while overall accuracy increases, perceptual capability barely changes — meaning almost all gains come from reasoning-side optimization, while the perception bottleneck is masked by the training signal.

Implications for Multimodal Training Methodology

From a broader perspective, SPLIT-RL points toward an important research direction: when a task involves multiple heterogeneous capabilities, a coarse reward signal may mask the true capability bottleneck. Decoupling learning signals for different capabilities through staged training and fine-grained advantage decomposition is a viable path toward improving the overall capability of multimodal models.

For researchers and engineers working on multimodal large model training, this work is a reminder that when designing reinforcement learning rewards, it's not enough to watch whether the final metric goes up — you also need to ask which sub-capability the improvement is actually coming from. Otherwise, you risk a kind of "false progress" where the score rises but a core capability remains stuck in place.

It should be noted that this article is based on the arXiv preprint abstract. Specific details about reward function design, ablation studies, and the implementation of CLA-GRPO's claim decomposition can be found in the full paper.