DoubtLess: Training-Free Contrastive Decoding That Helps Reasoning Models Think Less and Answer Right

Reasoning models excel at math, science, and coding tasks thanks to long chain-of-thought (CoT) reasoning, but the cost is generating large amounts of redundant tokens. A 32B model solving a competition problem might spend twice the necessary thinking time—rechecking, second-guessing, and verifying back and forth. Existing approaches to shortening CoT either require training per model, overlay fixed steering vectors on hidden states, or use crude decoding heuristics like "ban the word 'wait'"—often at the cost of accuracy.

A recent work called DoubtLess proposes a training-free contrastive decoding method: use a small reasoning model to guide a large model toward conciseness, without sacrificing accuracy.

How DoubtLess Works

The core mechanism is surprisingly simple. Have a small reasoning model read the CoT content generated so far by the large model, twice:

• Brief thinker: prompted to reason concisely
• Verbose thinker: prompted to reason verbosely

The difference between the logit vectors produced under the two prompts, multiplied by a strength coefficient, is added to the main model's logits at each decoding step. Intuitively, this difference captures the direction between "concise" and "verbose," pushing the large model's generation toward the concise side. The entire process requires no training and no modification to model weights.

Compared to methods like "ban the word 'wait'," DoubtLess's advantage lies in being a continuous, directional guide rather than a hard cutoff of certain tokens. The model can still express itself freely—it simply leans overall toward a more refined reasoning path.

Contrastive decoding is the key technical foundation here. Its core idea is to run a "strong" model and a "weak" model simultaneously (or the same model under two different prompts), then subtract the logits (i.e., the unnormalized scores over the vocabulary for each token) of the two outputs, amplifying the strong model's distinctive tendencies relative to the weak one. In DoubtLess, what's being contrasted is not a strong versus weak model, but rather the behavior of the same small auxiliary model under two prompts—the brief thinker and the verbose thinker. This way, the signal carried by the logit difference is a pure style/length direction, not a capability difference. Contrastive decoding works because a model's output distribution can be approximated as a superposition of "base tendencies" and "prompt-induced shifts." Subtracting cancels out the shared base tendencies, leaving only the difference dimension you want.

Measured Results: 31% Shorter, Same Accuracy

The paper validated the method on seven benchmarks covering competition math, graduate-level science, and competitive programming. Taking Qwen3-32B as an example:

• CoT length shortened by 31%, with accuracy unchanged
• Outperforms five inference-time baseline methods
• In a self-debugging coding agent scenario, thinking tokens per turn reduced by 20%, with no change in task performance
• Generalizes to two other model families, also without accuracy loss
• When combined with training-based compression methods, it can further shorten the CoT of models already trained for conciseness

These results show that DoubtLess is not a trick dependent on a specific model architecture, but a transferable decoding strategy.

Self-debugging coding agents refer to agent loops where the model, after generating code, runs it itself, observes errors or test failures, and then revises the code based on feedback. Each round of "generate—execute—revise" is a complete inference call, in which the model outputs a CoT to plan its revision strategy. Since such tasks often require multiple iterations, per-round CoT reduction compounds linearly with the number of turns—so even a 20% reduction in thinking tokens per turn translates into a substantial drop in end-to-end total token consumption and latency. This is why DoubtLess is particularly attractive in such scenarios: it's not a one-time saving, but one that gets amplified repeatedly through the loop.

Why It Works: Reducing Redundant Checking

Ablation experiments and behavioral analysis reveal a key pattern: the tokens DoubtLess saves come primarily from after the correct answer first appears.

A typical problem with reasoning models is "already got it right but can't stop"—continuing to re-verify, check from different angles, and second-guess. DoubtLess, through its concise-direction guidance, suppresses this post-hoc redundancy while having little effect on the exploration process before the answer first appears. This also explains why accuracy is preserved: the model isn't being deprived of thinking space—it simply stops over-deliberating.

The phenomenon of reasoning models "already got it right but can't stop" is closely tied to reward design in reinforcement learning training. When a model is rewarded for "getting the final answer right," it learns a conservative strategy: verify several more times to reduce the probability of error, even when the answer is already determined. This behavior is harmless or even beneficial within the training distribution, but at inference time it becomes pure waste. Mechanistically, this resembles an "uncertainty threshold" inside the model—it cannot reliably judge whether it is already certain enough, so it tends to keep checking. DoubtLess's concise-direction guidance essentially artificially lowers the probability of the "keep checking" behavior at decoding time, without touching the model's ability to solve the problem itself. That's why it can cut tail-end redundancy without sacrificing accuracy.

Implications for the Inference Efficiency Race

Efficiency of reasoning models is becoming a deployment bottleneck. Shorter CoT means lower inference costs, faster responses, and serving more requests with the same compute.

DoubtLess's value lies in its zero training cost and plug-and-play nature. Teams don't need to retrain compression strategies for each new model, nor manually tune parameters to ban specific words. One small auxiliary model plus contrastive decoding logic can be reused across multiple model families.

Of course, the method has its boundaries: it relies on a small model capable of understanding the reasoning process as a "guide," and the strength coefficient needs to be adjusted for the scenario. The paper's code is open-sourced on GitHub, providing a foundation for further research and engineering deployment.

As reasoning models grow ever larger, how to make them "think less and answer more accurately"—DoubtLess offers a training-free approach worth watching.

One-line Summary

DoubtLess uses training-free contrastive decoding to guide large models toward more concise chain-of-thought, shortening CoT by 31% with unchanged accuracy.

Paragraph Summary

While long chain-of-thought improves accuracy in reasoning models, it brings substantial redundant tokens and deployment costs. Existing compression methods either require per-model training or sacrifice accuracy. DoubtLess proposes a training-free contrastive decoding method: a small auxiliary model reads the CoT already generated by the large model under "concise" and "verbose" prompts, and the difference between the two logits is added as directional guidance to the main model's decoding. On seven benchmarks, Qwen3-32B's CoT length is shortened by 31% with unchanged accuracy, outperforming five inference-time baselines; in a self-debugging coding agent, thinking tokens per turn are reduced by 20%. Behavioral analysis shows the savings come primarily from after the correct answer first appears, effectively suppressing redundant checking, and the method can be reused across model families.