Adversarial Speculative Reasoning: A New Decoding Approach for More Reliable Long-Horizon Agents

From "Accelerator" to "Precision Allocator": Rethinking Speculative Decoding

Speculative Decoding has long been regarded as a purely system-level optimization technique: a smaller, faster draft model pre-guesses multiple tokens, which the target model then verifies in parallel in a single pass, thereby accelerating large-model decoding without changing the output distribution. Research on this technique has almost universally focused on a single metric—the acceptance rate.

But a study from RSS offers a different view: on complex reasoning tasks for long-horizon agents, this "accelerator" perspective does not hold. The researchers measured how frontier open-weight models perform under different speculative decoding schemes and found that numerical noise accumulates within the verification window, and the draft acceptance results of the thinking process can alter the model's output distribution after 30 steps, significantly affecting the reasoning accuracy of long-horizon agents in a random direction.

This means speculative decoding is not just a matter of speed—it is actually quietly reshaping the model's reasoning trajectory.

Acceptance rate is the core metric of speculative decoding, referring to the proportion of candidate tokens generated by the draft model that pass verification by the target model. In standard speculative decoding, the target model performs parallel validation of the candidate sequence: accepted tokens are kept, rejected ones are resampled from the corrected distribution, theoretically guaranteeing that the output distribution matches that of the target model decoding alone. The higher the acceptance rate, the more tokens are "freely" verified per unit time, and the greater the speedup. Early research therefore mostly aimed to increase the acceptance rate—for example, by making the draft model's distribution closer to the target model's, or by using multiple draft models or tree-structured candidates. But the acceptance rate only characterizes "how many tokens are accepted," not "which tokens are accepted" or how numerical errors during verification affect subsequent sampling—which is exactly why its limitations surface in long-horizon reasoning scenarios.

How Noise Within the Verification Window "Derails" Long-Horizon Reasoning

To understand this problem, one must first look closely at the mechanistic details of speculative decoding. In the standard speculative decoding pipeline, the draft model generates several tokens consecutively to form a candidate sequence, and the target model then verifies these candidates in parallel within a verification window. Ideally, verification either fully accepts or rejects the entire candidate segment, leaving the output distribution unchanged.

However, the study points out that in long-horizon agent tasks, numerical noise within the verification window grows progressively. Once the reasoning chain extends to dozens of steps, this noise can no longer be ignored: the draft acceptance results inadvertently alter the target model's output distribution. For an agent that must make dozens of consecutive decisions, a tiny shift at any step can be amplified by subsequent steps, ultimately steering the reasoning in a completely different direction.

This is also why the traditional acceptance-rate optimization approach fails in agent scenarios—no matter how high the acceptance rate, if key reasoning steps are "noise-ified" within the verification window, the overall success rate of the long horizon will still suffer.

Verification window refers to the interval of candidate tokens that the target model verifies in parallel in a single pass during speculative decoding, whose length is typically determined by the number of draft steps. Within the window, the target model computes conditional probabilities for each position and compares them with the draft distribution, deciding which tokens to keep according to accept-reject rules. Due to floating-point operation order, batch reduction, and the nondeterminism of parallel computation, the logits at each position within the window carry tiny numerical noise; when the window is long or the model is large, this noise can change the accept/reject boundary decisions, causing the actual sampling distribution to deviate from the theoretical one. In single-turn generation this deviation is usually negligible, but in agent tasks requiring dozens of consecutive decisions, it accumulates along the reasoning chain and gets amplified.

Doing the Opposite: Training the Draft Model to "Be Rejected"

The solution proposed by the research team is quite counterintuitive: since it is impossible to change the overall token distribution that the target model verifies, take a different angle—train the draft model to actively "be rejected" at key reasoning nodes.

Specifically, the approach first identifies the decision points in the agent's reasoning trajectory that are crucial to the outcome, then trains the draft model to form an adversarial preference: at these key points, the draft model's output diverges from the target model's, thereby triggering a verification restart. In this way, the key thinking tokens are resampled and encoded by the target model under the least noisy conditions, rather than being "glossed over" by the draft model's guesses.

In the researchers' words, this approach allows key reasoning to be "better indexed and more persistent over the long horizon." It does not pursue a higher acceptance rate; instead, it turns the speculative decoding mechanism into an active allocator of context precision.

Adversarial preference here refers to a training objective: making the draft model's output distribution at key decision points deliberately deviate from the target model's, so as to more easily trigger verification rejection and resampling. Conventional preference optimization brings the draft model closer to the target model to increase the acceptance rate, whereas adversarial preference does the opposite, treating "being rejected" as a signal so that tokens at key positions are regenerated by the target model under less noisy conditions. In practice, this usually requires first locating key reasoning nodes (such as tokens involving intermediate conclusions, tool-call parameters, or branch decisions), then constructing preference pairs for training. The cost is a lower overall acceptance rate and slower decoding, but what is gained is high-precision encoding of key context by the target model.

No Changes to the Target Model, Yet Higher End-to-End Success Rate

A key advantage of this adversarial speculative reasoning method is that the target model itself and its output distribution need not be changed at all. All modifications are concentrated on training the draft model, and the number of trained parameters accounts for only 0.06% to 0.28% of the hosted parameters.

On complex agentic coding benchmarks, this method improves the end-to-end success rate. This is noteworthy because it means the team has found a path to "achieving much with little"—without retraining or fine-tuning the large model, a tiny draft model alone can improve the reasoning accuracy of long-horizon agents.

For teams actually deploying agents, this offers a cost-controllable optimization option: rather than waiting for a larger, stronger target model, it is better to revisit the draft model's training objective and bring factors beyond the acceptance rate into the optimization scope.

Hosted parameters refer to the target model parameters that are hosted by the service provider at deployment and actually loaded during inference. In the study, "the number of trained parameters accounts for only 0.06% to 0.28% of the hosted parameters" means that only a tiny draft model or adapter module is trained, without touching the target model weights. This makes the method plug-and-play for existing open-weight models: the target model and its output distribution remain unchanged, and deployers only need to replace or attach the draft model, with training and storage costs far lower than full fine-tuning. However, the draft model still needs to align with the target model's tokenizer and vocabulary, and its adversarial preference needs to be retrained for the specific task distribution, with generalization depending on the accuracy of identifying key decision points.

Implications for Agent Engineering

The insights from this study go beyond speculative decoding itself. It reminds us that the reliability of long-horizon agents is a systems engineering problem, and details at the decoding level also affect the final outcome. When an agent needs to execute dozens or even more consecutive decisions, every link in the reasoning chain—including seemingly neutral decoding acceleration techniques—can become a source of error.

Another direction worth considering is "precision allocation": not all tokens are equally important, and key decision points deserve to be handled with higher precision and less noise. Adversarial speculative reasoning is a concrete implementation of this idea, shifting compute from "uniform acceleration" to "fine-grained sampling at critical points."

Of course, this method is currently still limited to validation on specific benchmarks, and its generality across broader agent tasks remains to be seen. But it at least opens up a new problem space: besides the acceptance rate, what other dimensions of speculative decoding parameters are worth optimizing?