Hands-On with Qwen 3.8 Max: How an Open-Source Model Closes In on Top-Tier Closed Models

The Soon-to-Be-Open-Sourced 2.4-Trillion-Parameter Behemoth

Qwen (Tongyi Qianwen) has just released its brand-new Qwen 3.8 Max model, and the specs are staggering: this is a massive model with a whopping 2.4 trillion parameters, and the team has officially confirmed it will be released as open-weight.

It's worth noting that behind this astonishing 2.4-trillion scale typically lies a Mixture of Experts (MoE) architecture for compute efficiency. The core idea of MoE models is this: although the total parameter count is enormous, only a small fraction of the "expert" sub-networks are activated during each inference pass. This keeps the actual computation within an acceptable range while preserving the vast knowledge capacity of a super-large model. MoE models represented by GPT-4 and Mixtral have already proven the viability of this approach, and it's the technical premise that allows Qwen 3.8 Max to serve requests with relatively reasonable latency.

The history of MoE architectures traces back to the "mixture of experts" neural networks proposed by Jacobs et al. in 1991, but they truly entered the large-model era with Google's 2017 "Sparsely-Gated MoE." The key breakthrough was the introduction of a learnable router network: for each input token, the router computes a matching score against every expert, selects only the Top-K experts for forward propagation, and completely skips the computation of the remaining experts. This "conditional computation" paradigm decouples model parameter scale from computational cost—the parameter count determines the upper bound of the model's knowledge capacity, while the actual inference FLOPs are determined solely by the number of activated experts.

Digging deeper, the design of the number of "experts" and the activation ratio in an MoE architecture is a core engineering decision. Take Mixtral 8x7B as an example: its total parameters are around 46.7B, but each token actually activates 2 experts, with an effective compute equivalent to a dense model of roughly 12.9B. If Qwen 3.8 Max's 2.4 trillion parameters follow a similar ratio, the actual activated parameters during inference may be only in the hundreds of billions, making it possible to serve reasonable latency on high-end server clusters. Another key engineering challenge of MoE is "expert load balancing"—if all tokens tend to activate the same few experts, some experts become overloaded while others sit idle. Modern MoE models typically introduce an auxiliary loss function to enforce uniform routing, which is an important guarantee that a 2.4-trillion-parameter scale can be realized in practice. Representative works like DeepSeek-MoE and Mixtral further introduce a shared expert mechanism, designating some experts as a "general knowledge base" that all tokens must activate, while the remaining experts handle domain-specific knowledge. This hierarchical design improves knowledge-sharing efficiency while preserving specialization capabilities.

However, at ultra-large scale, MoE architectures face an engineering challenge rarely discussed in public: cross-device expert communication overhead. At the scale of 2.4 trillion parameters, thousands of experts must inevitably be distributed across hundreds or even thousands of GPUs or TPUs. Whenever the router selects an expert for a token, if that expert happens to reside on a different compute node, cross-node data transfer must be triggered—a process that relies on the All-to-All communication primitive, introducing microsecond- to millisecond-level additional latency over InfiniBand or NVLink interconnect networks. Google's Switch Transformer paper systematically quantified this cost: in a 128-expert configuration, communication overhead can account for 15%–30% of total inference time. To mitigate this, the industry has developed hybrid strategies combining "Expert Parallelism" and "Data Parallelism," as well as locality-based expert placement algorithms—clustering experts that are frequently activated by the same class of tokens onto the same node to reduce cross-node routing frequency. This also means that the design of Qwen 3.8 Max's inference cluster is itself a complex systems engineering undertaking, and its real-world serving latency depends heavily on how well the underlying interconnect network topology is optimized.

In addition, there is an important distinction between "open-weight" and "fully open-source": open-weight models only release the trained weight files, allowing users to download, deploy, and run inference, but typically without accompanying complete training data or training scripts; fully open-source, on the other hand, requires that the training data and training code also be made public. Even so, open-weight is enough to enable the open-source community to do plenty of valuable work such as fine-tuning, quantization, and private deployment—for research institutions and enterprises with sufficient compute, it means being able to use a top-performing model without any API call fees.

At present, the weight files have not yet been officially released, and the full technical specifications and official benchmarks are still forthcoming, but the "open-source" part is a done deal. In its release announcement, the team made a bold claim—that it is one of the most powerful models around today, second only to Fable 5. Such a statement is rare in the fiercely competitive large-model arena, and it naturally invited hands-on scrutiny from the community.

Worth mentioning: you don't have to wait for the weights to be released to try it out. A preview version of Qwen 3.8 Max has already landed in the official token plans and can be called from platforms like Coder and CoderWork—subscribers can start using this cutting-edge model immediately at no extra cost.

Eight Hands-On Tests: From Elevator Simulation to 3D Watch

This evaluation uses the KingBench question set, covering front-end and animation tasks, Three.js scenes, SVG drawing, mathematical reasoning, long-horizon autonomous Agent tasks, and one extremely difficult 3D modeling problem. Each question is scored out of 10, and the model is invoked through Cloud Code (quad-code).

What sets KingBench apart from traditional academic benchmarks (such as MMLU and HumanEval) is the comprehensiveness and contamination-resistance of its tasks. Data contamination is a core problem in today's AI evaluation field—because the questions in mainstream benchmarks exist in vast quantities across the internet, they are very likely to have been included in a large model's pre-training corpus, causing models to "memorize the answers" rather than genuinely reason. Researchers have repeatedly demonstrated that by slightly rewriting the original questions, top models' scores drop significantly. This problem is known in academia as "benchmark saturation"; MMLU was maxed out by several models within a few years of its release, and math benchmarks like GSM8K face similar dilemmas.

KingBench's response strategy is to construct tasks with high combinatorial complexity: a single "3D watch" problem simultaneously tests Three.js rendering, time logic, animation interpolation, and interface design—requiring the model to handle the WebGL rendering pipeline, time calculations with JavaScript's Date API, CSS animation interpolation curves, and responsive UI layout all at once. Such multidimensional combinations have almost no exactly-matching samples in public datasets, making them very hard to answer through simple memorization. This approach aligns closely with the "dynamic benchmark" research direction that has emerged in academia in recent years—Stanford's HELM framework, DeepMind's BIG-Bench Hard, and even the "private red-team test sets" used internally at OpenAI are all exploring how to continuously produce "unseen" test samples through programmatic generation, combinatorial sampling, and similar methods, fundamentally breaking the limitations of static question sets. KingBench's engineering task design goes a step further: a task's "correctness" is no longer judged by a fixed answer, but evaluated through functional verification of the run result. This means that even if the question description leaks, the model must genuinely "implement" the functionality to score, rather than merely reproducing memorized code snippets. The evaluation uses the Cloud Code environment, meaning the model must complete tasks in a real tool-invocation environment rather than merely outputting static code strings, placing higher demands on the model's instruction-following and multi-turn interaction capabilities.

Front-End and 3D Interaction Capabilities

Question 1 · Elevator Simulation: Build a simulation system that can spawn people on different floors, with three elevators each carrying one passenger and remaining passengers waiting for the next one, plus a hover tooltip showing the destination floor. Qwen 3.8 Max scored 8 points—the elevator logic was correct and the animation smooth, on par with GPT-5 and GLM-5.2.

Question 2 · Contact Lens Case: A problem that trips up nearly every model, requiring a 3D model with distinct L/R lids that can be clicked open. Qwen 3.8 Max again scored 8 points, the second-best result the evaluator has ever tested—note that Opus 4.8 only scored 7 on this one.

Question 3 · Folding Table: Control the table's unfolding and folding via a slider, with seamless animation. It scored 8 points, tied with Opus 4.8 and only slightly below Fable 5 and Kimi K3, which scored 9.

SVG and Game Physics

Question 4 · Panda Eating a Burger SVG: Correct proportions, a recognizable burger—scored 8 points, tied for the highest with Kimi K3, Grok 4.5, and GLM-5.2, while Opus 4.8 only scored 6 here.

Question 5 · Archery Shooting Game: Four targets and a leaderboard mechanism ranked by shortest completion time. Qwen 3.8 Max earned a perfect 10—good physics feel and a properly updating leaderboard, a level only Opus 4.8 and Grok 4.5 had previously reached.

Surprises in Reasoning and Autonomous Agent Capabilities

If the front-end capabilities were somewhat expected, then what truly impressed about this model was its reasoning and long-horizon task capabilities.

Question 6 · Permutation and Combination Math Problem: This tricky problem of computing the number of ordered-pair arrangements has a correct answer of 2460, and many models got it wrong outright. Qwen 3.8 Max answered it exactly, scoring a perfect 10, joining the ranks of Opus 4.8, Fable 5, Kimi K3, and GPT-5.6. This proves it's not just a "pretty-face model" good only at front-end work—its underlying reasoning ability is genuinely solid.

Question 7 · Long-Horizon Autonomous Task: Autonomously generate a panda-knowledge dataset, fine-tune a Gemma 2B model, and build a local web UI that randomly displays panda facts on each refresh—all done automatically from scratch. The model completed it flawlessly, earning another perfect 10—three consecutive perfect scores, which is nothing short of stunning.

This problem belongs to the core application scenario of Agentic AI, which differs from traditional "Q&A-style" large models. Agentic AI systems are typically built on the ReAct (Reasoning + Acting) framework—jointly proposed by Princeton University and Google, in which the model iteratively advances a task through a loop of "Thought → Action → Observation." Unlike single-turn Q&A, errors in Agent tasks have a cascading effect: a wrong file path generated in step 3 can cause a read failure in step 7, and the model must recognize this cross-step dependency at runtime and backtrack to fix it. The industry currently defines "long-horizon planning ability" as the success rate on tasks exceeding a 12-step tool-call chain, and this metric is often a better predictor of a model's real-world production value than single-turn ability. The Tool Use Layer is another key component—the model must accurately map natural-language intent into structured function calls (such as the OpenAI Function Calling or Anthropic Tool Use format), and any parameter error or hallucination will cause the tool execution to fail. This is why long-horizon Agent tasks are widely recognized in the industry as the key litmus test distinguishing "truly intelligent" models from "superficially fluent" ones—they require the model to autonomously plan multi-step action sequences, call external tools (file system, shell commands, Python runtime), dynamically adjust strategy based on intermediate results, and self-correct when errors arise. Any failed tool call or logical break can bring the entire task chain crashing down.

From a more macro perspective, Qwen 3.8 Max's perfect score on Question 7 also reflects the current industry trend of "the model capability boundary shifting toward the system integration layer." Early Agent frameworks (such as LangChain and AutoGPT) required engineers to manually orchestrate tool-call chains, with the model itself only responsible for filling in the parameters of each step. But when a model is capable enough, the role of this "scaffolding" gradually weakens—the model can directly understand a task's full dependency graph, autonomously decomposing sub-tasks, selecting tools, and handling exceptions without an external orchestration framework. This is precisely the deeper logic behind Anthropic defining the core capability of its Claude series as "Computer Use" and OpenAI strengthening tool-call stability in GPT-4o: the competitive focus of next-generation AI is shifting from "single-answer quality" to "multi-step task completion rate."

The Gemma 2B fine-tuning involved typically uses parameter-efficient fine-tuning techniques such as LoRA (Low-Rank Adaptation). LoRA was proposed by Microsoft Research in the 2021 paper LoRA: Low-Rank Adaptation of Large Language Models, and its core insight is that when a large pre-trained model adapts to a downstream task, the weight change matrix has an extremely low intrinsic rank. Therefore, it can be decomposed into the product of two low-rank matrices $\Delta W = BA$, where $B \in \mathbb{R}^{d \times r}$, $A \in \mathbb{R}^{r \times k}$, and the rank $r \ll \min(d, k)$. Take Gemma 2B as an example: a certain layer's fully connected weights might be a 4096×4096 matrix (about 16 million parameters), while the corresponding LoRA matrices at rank r=16 require only about 130,000 parameters, a compression ratio exceeding 100x. During training, LoRA freezes the original pre-trained weights and only trains the newly introduced low-rank matrices (usually less than 1% of the original parameter count), drastically reducing memory and time costs and making it feasible to complete domain fine-tuning of billion-parameter models on a single consumer-grade GPU (such as the RTX 4090). QLoRA builds on this by further introducing 4-bit quantization, compressing a 24GB memory requirement down to about 6GB and lowering the fine-tuning barrier even further. Being able to autonomously complete the full pipeline of "generate training data → run fine-tuning → deploy and display" means the model not only understands the abstract logic of the machine-learning workflow but can also correctly call the APIs of specific libraries like Hugging Face Transformers and PEFT—directly valuable to the day-to-day productivity of AI engineers.

Question 8 · 3D Watch: The hardest problem in the entire question set, requiring a complete 3D watch with real-time timekeeping, smooth motion of the second/minute/hour hands, day-of-week switching, and dual time zones. Qwen 3.8 Max scored 3 points—the watch renders and the hands move, but it failed to fully implement all requirements. In the context of the entire leaderboard, the highest score on this problem is Fable 5's 4 points, and many models scored a flat 0, so 3 points is actually a pretty decent attempt.

Final Ranking: Second Only to Fable 5, Ahead of Opus 4.8

Across all eight questions, Qwen 3.8 Max scored 65/80 (81.25%), placing second on the entire leaderboard.

• Fable 5: 82.5%
• Qwen 3.8 Max: 81.25%
• Opus 4.8: 80%
• Kimi K3: 77.5%
• GLM-5.2: 75%
• GPT-5.6: 71.25%

The official announcement's claim of being "second only to Fable 5" was almost precisely validated by this independent evaluation—quite rare in an era where marketing rhetoric tends to exaggerate.

Real-World Usage Experience and Shortcomings

Beyond the leaderboard, the evaluator also shared their real-world impressions. When called within quad-code, the model responded quickly and followed instructions well, and the output quality did indeed reach the level of top closed-source models.

There is, however, one concern: during long-running tasks, the model's performance may not be stable enough—in longer sessions, it occasionally exhibits harness-level minor issues, such as failing to create a file when it should. This forms an interesting contrast with the stellar perfect score on the long-horizon Agent task in Question 7, perhaps reflecting the model's performance differences across different ranges of task complexity, or possibly stemming from the current version not yet being fully tuned for the quad-code environment. Notably, such instability is not uncommon in MoE-architecture models—the randomness of the routing mechanism and the behavioral differences of various experts at the tail of the context window can both cause occasional fluctuations in output quality during long sessions. This phenomenon is known in the industry as "context drift": as session length grows, the attention mechanism's weighting of early key information gradually decays, and MoE's dynamic routing may activate expert combinations mismatched to the task at the tail end of a long context, further exacerbating this instability.

From a technical-mechanism standpoint, context drift has a special amplification mechanism in MoE architectures. In a standard Transformer, the decay of attention weights is a relatively continuous process; but in an MoE model, routing decisions are discrete—the same token might be routed to a "code expert" when the context length is 2K, but switch to a "general text expert" when the context length reaches 8K, because the distribution of surrounding tokens changes. Such routing jumps introduce discontinuous capability switches over longer sessions, manifesting as the model "suddenly forgetting" the previously established code context or tool-call conventions. Current industry mitigation approaches include: fixing the routing results of some experts at inference time (Sticky Routing), introducing position-encoding-based routing bias, and periodically restating key constraints in the system prompt. These engineering optimizations are often already built into closed-source API services, but in self-hosted scenarios of open-weight models, how to configure these parameters remains an engineering detail that users must actively explore. The evaluator stressed that this is only a preliminary observation and no final conclusion can yet be drawn.

A Major Gift to the Open-Source Community

All things considered, Qwen 3.8 Max is a rather excellent model: second on the leaderboard, edging out Opus 4.8, and scoring perfectly on all three categories of game physics, mathematical reasoning, and autonomous Agent tasks. And its most enticing aspect is accessibility—the preview version is already included in the token plans, letting subscribers use a frontier-level model at a cost far below that of closed-source labs.

Add to that the imminent release of the weights in open-weight form, and this is undoubtedly a major boon for the entire open-source community. An open-weight model with 2.4 trillion parameters, using an MoE architecture to keep the actual activated parameters within a deployable range, and reaching this performance level, is certainly not an opportunity that comes along every day. The downstream ecosystem around open weights is already quite mature: research institutions can perform domain fine-tuning based on LoRA, enterprises can privately deploy the model on their intranets to avoid data-compliance risks, and the GGUF quantization pipeline aimed at consumer-grade hardware (led by the llama.cpp project, which supports compressing model weights to 2–8 bit integer representations) allows ordinary developers to run billion-parameter-scale models locally.

It's worth specifically explaining what GGUF quantization means for a 2.4-trillion-parameter MoE model. Even with aggressive 2-bit quantization, 2.4T parameters still require about 600GB of storage—far beyond the memory ceiling of a single consumer-grade server. Therefore, a more realistic path for "local deployment" of Qwen 3.8 Max may be expert subset quantization: loading and quantizing only the subset of experts frequently activated by specific domain tasks, deploying it as a lightweight "domain-specialized distilled version" rather than the full 2.4T-parameter model. This approach aligns closely with the "on-demand expert loading" schemes that companies like Huawei and Lenovo are exploring in edge-AI scenarios, and it foreshadows the innovation direction the open-source community will unleash once it gets its hands on the weights: no longer "getting the full model running," but "precisely distilling domain capabilities out of a giant model." How it truly performs in long-horizon, complex, real-world scenarios still awaits longer-term validation, but based on the current report card, Qwen 3.8 Max is already exciting enough.

Key Takeaways