FADE: A New Online RLVR Scheduling Method that Decouples Ability and Difficulty

Problem Background: When the Pass-Rate Tracker "Loses Focus"

In RLVR training, only prompt groups whose sampled completions disagree can produce a non-zero relative advantage. Groups that are all-pass or all-fail contribute nothing to the policy update yet consume generation budget. Adaptive schedulers therefore try to predict which prompts are more likely to yield "non-degenerate" sample groups, but existing methods usually maintain a pass-rate tracker per prompt, such as an exponential moving average or a Beta counter.

The problem is that a prompt's pass rate is simultaneously affected by two factors: changes in the model's ability on the skills that prompt requires (which can drift quickly), and the prompt's own intrinsic difficulty (which is relatively stable). A single tracker cannot capture both time scales at once—short memory causes the relative ranking among prompts to fluctuate wildly, while long memory reports stale pass rates. The FADE authors point out that this entanglement makes scheduling decisions suboptimal and wastes the training budget.

RLVR (Reinforcement Learning with Verifiable Rewards) is a paradigm widely adopted in recent years for large-model reasoning training: multiple reasoning trajectories are sampled for each prompt, a binary reward is assigned using automatically verifiable rules (such as matching math answers or passing code unit tests), and policy gradients are then constructed via within-group relative comparisons (e.g., the group-normalized advantage in GRPO). Because an advantage is only meaningful when the sampled results within a group differ, a prompt whose sampled trajectories are all correct or all wrong contributes zero gradient. This is precisely the root of wasted sampling budget: easy prompts in early training are often dominated by "all-correct" groups, while extremely hard prompts remain "all-wrong" for a long time, and neither provides an effective learning signal. Understanding this mechanism helps clarify why a scheduler needs to predict the probability of "non-degenerate groups" arising, and why the accuracy of pass-rate estimation directly determines training efficiency.

FADE's Core: Bilinear State-Space Filtering

FADE models each prompt's success log-odds as a combination of two parts: the inner product of a whitened K-dimensional skill loading and a fast-drifting policy ability vector, minus a slowly shrinking scalar item intercept. In formula form, a prompt's success logit = skill loading · ability vector − item intercept. Here, the ability vector reflects the model's current level across different reasoning skills, while the item intercept captures the prompt's intrinsic difficulty.

The key to this decomposition is that the skill loading is fixed (extracted from the prompt content), whereas the ability vector updates quickly over time and the item intercept changes slowly. Through Pólya–Gamma augmentation, each mini-batch of binary sampling outcomes yields a closed-form update of the K-dimensional ability posterior; using a rank-K Woodbury step, changes in ability are propagated instantly to all unobserved prompts that share the same skills, while keeping the item intercept stable.

From the updated posterior mean and variance, FADE derives a closed-form utility function for estimating the probability of drawing a non-degenerate sample group. The scheduler uses it to select more informative prompts, improving training efficiency.

Pólya–Gamma augmentation is a sampling trick used for Bayesian logistic regression: it represents the log(1+e^x) term in the logistic likelihood as a mixture of a Gaussian variable and a Pólya–Gamma random variable, thereby giving the originally non-conjugate binary observation model a conditionally conjugate structure, so that closed-form Gaussian posterior updates can replace MCMC. FADE leverages exactly this trick to turn each prompt's success/failure samples into linear Gaussian observations of the K-dimensional ability vector, enabling mini-batch updates to complete in milliseconds. The Woodbury matrix identity is used to compute low-rank updates efficiently: (A + UCV)^{-1} can be expressed via A^{-1} and the inverse of a small-scale matrix, reducing repeated O(K^3) inversions to the O(K^2) order—this is precisely the key to FADE keeping the extra overhead below 0.6 milliseconds.

Experimental Results: Gains Across Benchmarks and Scheduling Paradigms

The paper evaluates on five reasoning benchmarks, two base models, and four scheduling paradigms, with all experiments conducted under matched generation budgets. The results show that FADE consistently improves downstream accuracy and calibration, while the per-step extra overhead is below 0.6 milliseconds and thus almost negligible.

You may not have noticed that FADE's advantages hold across different scheduling paradigms, indicating that its idea of decoupling ability and difficulty is universal. Compared with baselines that rely on a single pass-rate tracker, FADE can more accurately identify prompts whose model ability is changing but which have not yet been sufficiently explored, thereby generating more informative sample groups.

Calibration measures the consistency between the model's output confidence and the actual correctness rate: if the model claims 80% confidence on a batch of prompts, then the actual pass rate on that batch should be close to 80%. In RLVR, poor calibration means that the probabilities given by the pass-rate tracker deviate from the true success rate, so the scheduler's judgment of "which prompts are more informative" will be systematically wrong. Commonly used evaluation metrics include expected calibration error (ECE) and the Brier score. FADE improves calibration while raising accuracy, indicating that the non-degenerate-group probability estimates given by its utility function are not only better ranked but also more trustworthy in absolute terms, which is especially important for scheduling strategies that need to set sampling thresholds.

Implications and Outlook

The value of FADE lies in introducing the idea of ability estimation from meta-learning into RLVR scheduling, explicitly separating the two confounding factors of "prompt difficulty" and "model ability." This decomposition not only improves scheduling efficiency but also provides interpretable intermediate variables for understanding how the model's skills evolve during training. In the future, the method may be extended to more complex reward structures or multimodal tasks, further unlocking the training potential of online RLVR.

For engineers working on large-model reasoning training, FADE offers a lightweight plug-and-play solution—without modifying the existing RLVR pipeline, more stable training signals can be obtained simply by improving the scheduler. As reasoning models increasingly demand data efficiency, fine-grained scheduling strategies of this kind will become more and more important.