Qwen3.5 Technical Deep Dive: How 5% Active Parameters Outperform Billion-Scale SOTA Models

Alibaba's latest release, Qwen3.5, has sparked intense discussion in the tech community. This MoE model—with 397 billion total parameters but only 17 billion activated per inference—achieves less than 5% activation rate while surpassing previous-generation hundred-billion-scale models in performance. Based on hands-on testing shared by a tech creator on Bilibili, this article breaks down the three core technologies behind it: hybrid attention, ultra-sparse MoE, and multi-token prediction.

Qwen3.5 Model Overview: Small Activation, Big Capability

Qwen3.5 is a Vision-Language Model (VLM) that accepts images, pictures, and video as input and outputs text, comparable in capability to the previous VL series models. In terms of parameter structure, it's an MoE architecture with 397B total parameters and 17B activated per inference—an activation rate of roughly 4.3%, which is its most striking feature.

Compared to the ~10% activation rate of most previous MoE models, Qwen3.5's sparsity level is remarkably aggressive. Its biggest selling point: dramatically improved efficiency with virtually no sacrifice in performance. According to hands-on testing, this model leads in three key dimensions: Coding, Math, and long-context processing.

Why these three dimensions? Coding is currently the most widely deployed and easily monetized AI application scenario; math tests whether a model can push beyond the limits of human intelligence; and long-context capability is essential for agent applications—especially in agent frameworks like Claude's that rely heavily on tool calling, where long-context processing is critical. Additionally, Qwen3.5 expands language support to 201 languages, including less common ones like Filipino and minority Indian languages, demonstrating clear international ambitions.

Breaking Down Qwen3.5's Three Core Technologies

Hybrid Attention Mechanism: Three Linear Layers for Every Traditional Layer

Qwen3.5 isn't the first to propose hybrid attention—this architecture was foreshadowed in earlier "Qwen NAS" research (NAS hinting at next-generation architecture). Hybrid attention introduces Linear Attention on top of the traditional Transformer framework.

Traditional attention has computational complexity of N², growing quadratically with token length; linear attention is only N, offering a full order-of-magnitude speedup. However, linear attention incurs some information loss, so Qwen3.5 adopts a compromise: three layers of linear attention paired with one layer of traditional attention.

This design yields several key benefits:

• Lower VRAM usage during inference: Linear attention only needs to cache the current state—KV Cache usage is independent of sequence length, while traditional attention's cache grows linearly.
• NPU-friendly: Traditional attention is a "dynamic graph" where computation varies dynamically as tokens grow, requiring frequent scheduling changes on the chip. Linear attention has fixed computation like a "static graph," naturally suited for compute-intensive chips like NPUs. NPUs cost less, which is a significant advantage.
• Enhanced long-context extrapolation: The model also introduces a gating mechanism—a technique from last year's best paper at a top conference—used for secondary filtering of attention results. It adds nonlinear capability and eliminates the "Attention Sink" phenomenon, where tokens inexplicably show high similarity to the first few tokens as a conservative strategy. Once eliminated, long-context extrapolation capability improves significantly.

It's worth noting that linear attention is "context-dependent": when text is very short, the difference between N and N² is minimal, and highly optimized traditional attention may not lose out. But once text gets long, linear attention's advantage explodes immediately. The creator used an analogy of "electric cars vs. gas cars"—traditional attention is like a perfectly optimized gas car that hybrid attention can't beat yet, but as a new paradigm, its growth potential is enormous.

Ultra-Sparse MoE: Software Driving Hardware Development in Reverse

The 17B/397B activation ratio means less than 5% activation rate, which is extremely difficult to achieve in engineering. Traditional MoE models maintain ~10% activation rates because: too-low activation makes routing much harder (wrong expert selection is costly), computation becomes too sparse rather than dense (both GPUs and NPUs prefer dense computation), and communication pressure increases (frequent data exchanges).

Qwen3.5's willingness to push activation this low will likely catalyze inference chips designed specifically for sparse MoE. This also explains why Alibaba is developing its own T-Head (Pingtouge) chips—using software model development to drive specialized hardware in reverse, forming a strategic closed loop of "software pulling hardware forward."

Multi-Token Prediction (MTP): Built for Agent Scenarios

Traditional models decode by predicting tokens one at a time (T1–T4 predicting T5), but Qwen3.5 introduces Multi-Token Prediction (MTP): after predicting T5, a very small module (rather than running through the entire large model again) simultaneously predicts T6 and T7.

This aligns with human intuition—when you say "The capital of China is..." you can predict "Beijing" all at once rather than character by character. DeepSeek^Ad, Meta, and Google have all adopted this technique; MTP is becoming a standard capability for large models. This optimization primarily serves agent scenarios, significantly boosting decoding speed.

Qwen3.5 Performance and Cost: Where Does the 19x Inference Speedup Come From?

At 259k throughput, Qwen3.5 achieves approximately 19x speedup. Here's how that number breaks down:

• Hybrid attention (linear attention) contributes roughly 5x
• MoE sparsification contributes roughly 2x
• Multi-token prediction contributes roughly 1.5x

5 × 2 × 1.5 ≈ 15, plus other engineering optimizations, totaling approximately 19x. This means inference costs are roughly 1/19th of before.

Even more critical is "effective cost"—speed alone is useless if accuracy suffers. The creator offered an analogy: one student scores 700 on the college entrance exam but studies 12 hours a day; Qwen3.5 is the student who studies only 2 hours a day but scores 680. Though 20 points lower, it's clearly smarter and more efficient. While MoE, MTP, and hybrid attention optimizations could theoretically reduce accuracy, real-world testing shows the success rate remains very high.

Training Framework Innovation: Decoupled Training-Inference and Native FP8

Qwen3.5's capabilities are backed by a powerful training framework. It continuously collects data through Qwen agents, feeds it into task queues, and pulls directly during training—achieving a decoupled training-inference reinforcement learning framework that supports a replay system of trillions of tokens, enabling rapid adoption of new techniques.

Additionally, it employs a native FP8 pipeline. Last year, DeepSeek was the first to successfully implement FP8 half-precision training; a year later, Qwen3.5 has replicated this capability, saving nearly 50% of VRAM and further reducing training costs.

An interesting industry pattern: the Qwen team prefers using "elegant mathematics" to reduce costs, while DeepSeek excels at using "engineering prowess" (various low-level optimizations). Neither approach is inherently superior, and they'll likely learn from and merge with each other in the future. The fact that China can simultaneously produce both Qwen and DeepSeek—two cost-reduction-oriented models—is in some sense a result of being "forced" by chip restrictions. Without abundant compute resources, you have no choice but to push efficiency to the extreme.

Alibaba's Strategic Ambition: Full-Stack AI Layout

From a broader perspective, Qwen3.5 reflects Alibaba's strategic positioning. Alibaba is building a complete "AI stack": the Qwen model series at the top, Alibaba Cloud as the middle platform layer, and T-Head chips at the bottom. This stands in stark contrast to Tencent and ByteDance, who focus primarily on upper-layer applications—Alibaba's ambitions are clearly much larger.

For everyday developers, the most direct benefit is "cheap and fast." Qwen models come in a full range of sizes and are easy to deploy in production. A multi-agent stock sentiment and factor analysis system built on Qwen3.5 runs even faster than 7B small models. The AI industry has already shifted from "benchmark chasing" to "cost reduction and efficiency"—agent applications create unprecedented cost pressure, and whoever can keep costs down while maintaining capability holds the power.

Conclusion

Qwen3.5 surpasses billion-scale SOTA models with less than 5% active parameters—fundamentally, this is a victory in "parameter utilization efficiency." It proves that models don't need to keep getting bigger. Through the triple combination of hybrid attention, ultra-sparse MoE, and multi-token prediction, smart architectural design rather than brute-force parameter scaling can achieve a win-win on both performance and cost. For the exploding wave of agent applications, this kind of efficient, low-cost, long-context-capable model may be the most pragmatic choice.