Swift-Qwen3.8-27B: 58% Fewer Thinking Tokens, Nearly 2x Faster Inference

One of the hidden costs of AI reasoning models is the massive amount of redundant "thinking" content they generate while solving problems. Emerging lab UkisAI recently open-sourced Swift-Qwen3.8-27B on Reddit, claiming to have reduced thinking tokens by 58.3% and boosted inference speed by 1.95x through post-training techniques — all while keeping accuracy loss within 1%. The ideas behind this approach, and why it differs from typical "force-shorten reasoning" methods, are worth unpacking.

The Problem: It's Not That Thinking Is Too Long — It's "Overthinking"

While running quantized versions of Qwen3.8-27B, the UkisAI team repeatedly encountered a frustrating phenomenon: randomly occurring reasoning loops they dubbed "overthinking errors." These loops persisted even at low-to-medium reasoning intensity settings, manifesting as anxious, repetitive deliberation that appeared to be thinking but contributed nothing to the final answer quality.

The team emphasizes a core argument: reasoning length itself matters greatly and should be optimized, not forcibly shortened. This stands in sharp contrast to many "compress the chain-of-thought" approaches on the market. They argue overthinking is a problem independent of reasoning intensity settings, chat templates, or token limits — meaning Swift's training method complements existing reasoning effort mechanisms rather than replacing them.

Interestingly, the team initially assumed these loops were a side effect of low-precision quantization (PTQ), but experiments revealed the pattern was widespread even in BF16 full-precision models of the same size. This indicates a deeper root cause and broadens the applicable scope of the optimization approach.

Technical Approach: Identify and Penalize "Overthinking Tokens"

UkisAI's methodology draws inspiration from a Meta paper, though applying it directly yielded inconsistent results. The team realized the key was "locking onto the right keywords" and tuning hyperparameters. They spun up an 8×H100 cluster to generate large volumes of reasoning traces across multiple out-of-distribution (OOD) domains — including coding, language, vision, and agentic tasks.

They categorized traces exhibiting overthinking, identified the "common denominator" tokens among them, and built an inference-time penalizer targeting the most significant ones. This penalizer outperformed the original token set from the reference paper and proved effective not only on quantized models but on BF16 as well.

From Penalizer to LoRA Training

The initial plan — using the penalizer to generate traces and then running cross-entropy SFT — failed completely. The team pivoted to using the identified tokens to construct a loss function and ran LoRA SFT on the previously generated traces. This time, reasoning length dropped significantly and the reduction generalized — but accuracy fell along with it.

To recover accuracy, they tried several approaches: reinforcement learning (GSPO), On-Policy Distillation, and adapter block-merging from ThinkingCap 3.6 27B. Ultimately, accuracy loss was brought within 1% on nearly all internal OOD tests. The team considers On-Policy Distillation a key component of their "secret weapon."

Benchmarks: Tokens Cut in Half, Accuracy Largely Preserved

The team conducted rigorous benchmark evaluations, running each benchmark 10 times (5 for base, 5 with adapter), following the standard procedures from the Qwen 3.6 27B model card on Terminal Bench. Here are the key results at xhigh reasoning intensity:

Benchmark	Qwen3.8-27B	Swift-27B	Token Savings
GPQA-Diamond	88.4%	88.3%	58%
LiveCodeBench v6	76.8%	81.6%	46%
Terminal-Bench 2.1	66.7%	65.8%	39%
MMLU-Pro	85.5%	85.0%	28%
C-Eval	90.0%	90.6%	19%
IFBench	73.5%	71.8%	51%
AIME 2026	98.7%	94.0%	50%
HMMT	99.3%	96.0%	46%
ERQA (Vision)	67.5%	66.3%	55%

Accuracy fluctuations on most tasks stay within 1%, with some tasks even showing improvement. The one notable exception is AIME 2026, where accuracy dropped 4.6% — the team candidly attributes this to a training bug where a key math-reasoning token was mistakenly penalized, and they plan to fix it in a future release. This kind of transparent disclosure is commendable in an open-source project.

The Key Comparison: xhigh vs. medium

The most telling data point is the three-way comparison on GPQA-Diamond: base xhigh achieves 88.4% accuracy but consumes a median of 6,642 tokens; Swift xhigh holds 88.3% accuracy with tokens dropping to 2,771 — nearly cut in half; meanwhile, base medium uses fewer tokens (1,753) but accuracy falls to just 84.1%.

In other words, Swift maintains xhigh-level accuracy at less than half the token count, while outperforming base-medium by 4 percentage points at roughly 1.6x the token cost. This validates the team's core thesis of "optimize, don't truncate." That said, token savings vary across reasoning intensity levels: xhigh averages a 41% reduction, medium 23%, and low 26%, with medium and low settings showing 1–4% accuracy degradation that warrants further testing.

Ecosystem and Open Source: Model, API, and Licensing

UkisAI has fully open-sourced the model on Hugging Face, providing Q1–Q8 GGUF quantizations, with community contributions including Bartowski quants, NVFP4, W4A16, and even an uncensored version. With GPU support from Nvidia, the team also offers an OpenAI-compatible free research API, rate-limited to 5 RPM, for users with limited compute.

One licensing note: it's not Apache 2.0, but the team states the restrictions only apply to companies with annual revenue exceeding $1 million — a non-issue for most community users and researchers. As a new lab, they've also kept some training details and data private.

The team revealed they're developing Swift 3.8 Flash Next, which has already achieved -30% thinking tokens while maintaining xhigh accuracy — a signal they see as strong evidence their methodology is reproducible across the Qwen family, with plans to expand to other model families.

Key Takeaways

The most important insight from this work is a redefinition of "inference efficiency" — genuine optimization isn't about cutting out thinking, but about precisely removing the "anxious loops" that don't contribute to answer quality. This is fundamentally different from the token caps and forced truncation approaches common in the industry, which explains why Swift can substantially increase speed while preserving high-reasoning-intensity accuracy.

Of course, all current results are self-reported by the team, and independent community reproduction remains to be seen. The token mispenalization issue exposed on AIME also reminds us that token-level interventions carry precision risks. For developers deploying reasoning models locally and constrained by compute costs, Swift-Qwen3.8-27B is still a direction worth exploring hands-on.