Wukong 2.2P Open Source: 35B MOE Model Outperforms Qwen3.6-27B at 3–5x the Speed

Overview: A New Benchmark for Open-Source Small Models

The AI creator team "Shénqí Néng Yíng" has officially open-sourced their latest work — Wukong 2.2P (Plus), a 35B MOE model. Built on the Qwen3.6-35B base and trained using "adversarial hybrid distillation," this model claims to outperform the original Qwen3.6-27B in overall performance, while offering significant advantages in inference speed and VRAM usage. For users seeking a cost-effective local deployment solution, Wukong 2.2P is well worth paying attention to.

The open-source small model space is fiercely competitive, with an ecosystem dominated by major base models including Qwen (Alibaba), LLaMA (Meta), Mistral, and Gemma (Google). On top of these foundations, a large number of third-party teams have built derivative models through fine-tuning, distillation, and merging techniques, forming a vibrant model community. The "adversarial distillation" approach represented by the Wukong series follows a similar path to Nous Research's Hermes series and Teknium's OpenHermes — pursuing performance breakthroughs on open-source bases through innovative training methods. This community-driven model innovation is rapidly closing the gap with commercial closed-source models, while providing local deployment users with an ever-growing selection of high-quality options.

Core Technology: How Adversarial Hybrid Distillation Enables Small Model Breakthroughs

The Adversarial Hybrid Distillation Training Paradigm

Wukong 2.2P continues the core training methodology of the Wukong 2.2 series — adversarial knowledge distillation. Knowledge distillation was originally proposed by Hinton et al. in 2015, with the core idea of having a small model (student model) learn the output distribution of a large model (teacher model), rather than simply learning hard labels. Traditional distillation typically involves a one-to-one teacher-student relationship, whereas "hybrid distillation" introduces multiple teacher models, allowing the student model to draw knowledge from multiple sources — analogous to hybrid vigor in biology. The "adversarial" component borrows from the concept of GANs (Generative Adversarial Networks), using an adversarial training framework to push the model to continuously improve on more challenging samples. This prevents simple imitation of the average behavior of teacher models, enabling the student to surpass any single teacher on specific capability dimensions.

Wukong 2.2P and the earlier Wukong 2.2 Lite 9B are nearly identical in training methodology and dataset, with the sole difference being an upgrade of the base model from Qwen3.5-9B to Qwen3.6-35B. The larger base model combined with mature distillation techniques delivers a qualitative leap — the team claims its overall performance not only surpasses the original Qwen3.6-27B, but "far exceeds the original 35B model" as well.

MOE Architecture: The Fundamental Reason Behind Speed and Capacity

The key to understanding Wukong 2.2P's speed advantage lies in understanding the fundamental difference between MOE (Mixture of Experts) architecture and traditional dense models. Dense models activate all parameters during every inference pass, whereas MOE models contain multiple "experts"