Tencent Hunyuan V3 Open-Sourced: Full Breakdown of the 295B MoE Architecture — 256K Context Is Its Biggest Weakness

Tencent Hunyuan V3 Official Release Goes Open Source — 295B MoE Architecture Arrives

Tencent has officially open-sourced the latest version of its Hunyuan large language model — Hunyuan V3. Tencent had previously released a Hunyuan V3 Pro preview; this open-source release is the full, official model. Architecturally, Hunyuan V3 has a total of 295 billion parameters and is a Mixture-of-Experts (MoE) model with approximately 21B active parameters, complemented by a ~3.8B-parameter MTP module dedicated to inference acceleration.

On MoE Architecture: Mixture-of-Experts is a sparsely activated neural network architecture. Unlike traditional dense models that activate all parameters for every inference pass, MoE models divide the network into multiple "expert" sub-networks. During each forward pass, a Router dynamically selects a small subset of experts to handle the current input. The 295B total / 21B active parameter ratio means only ~7% of parameters are invoked per inference — significantly more compute-efficient than a dense model of equivalent scale. This is a core reason why leading LLMs have broadly adopted MoE architectures.

On the MTP Module: MTP (Multi-Token Prediction) is an inference acceleration technique. Traditional autoregressive language models predict only the next token at each step, whereas an MTP module can predict multiple future tokens in a single forward pass, using a verification mechanism to determine which predictions to accept — dramatically boosting token generation throughput. This is conceptually similar to Speculative Decoding, a mechanism also used by the DeepSeek^Ad series. Hunyuan V3's dedicated 3.8B-parameter MTP module signals that inference acceleration is a serious engineering investment for Tencent, making it especially well-suited for Agent scenarios that require high-frequency model calls.

Notably, after releasing the preview, Tencent reportedly collected feedback from over 50 products — much of it from their own Agent client product WorkBody (Tencent's coding/AI agent tool). By making the model freely available to a large user base and collecting the resulting data, Tencent obtained valuable real-world usage signals, which then fed into reinforcement training and optimization. The end result, as reflected in the official messaging, is that this upgrade focuses squarely on strengthening Agent capabilities.

Internal Benchmarks: Significant Agent Capability Gains

Setting aside comparisons with competitors for a moment, the improvements Hunyuan V3 shows over its own Pro preview are quite substantial — particularly on several key Agent-focused benchmarks:

• Terminal Bench (terminal operation capability)
• Skill Bench (tool/skill invocation capability)
• Crayfish (programming Agent scenarios)

These are exactly the metrics that matter most for evaluating a model's practical value in the Agent era. Tencent WorkBody enjoys enormous popularity among domestic AI agent client users, with its core advantages being ease of use and relatively accessible pricing. With such a large volume of real user data feeding back into training, Hunyuan V3's targeted optimization for Agent use cases comes as no surprise.

Head-to-Head Comparison: Claims to Surpass DeepSeek-V4-Pro

According to Tencent's internally published benchmark data, Hunyuan V3 outperforms DeepSeek-V4-Pro in nearly every category, with the only area where it falls slightly short being MCP-related tests. By the official account, the claim of "comprehensively surpassing DeepSeek-V4-Pro" is essentially unambiguous.

That said, it's important to view vendor self-reported scores with appropriate skepticism. Every company publishes benchmark results that favor their own model; real-world usability is what ultimately matters. From hands-on experience, DeepSeek-V4-Pro remains an exceptionally capable model across the board, and several models that previously claimed to surpass it have fallen short in practice. Hunyuan V3 hasn't undergone deep real-world testing yet, so it's too early to draw firm conclusions.

Beyond the DeepSeek comparison, Hunyuan V3 was also benchmarked against the GLM series. The takeaway: Hunyuan V3 surpasses GLM 5.1 but still trails GLM 5.2 in most areas — it can edge ahead in some domains, but is generally behind elsewhere, which is why Tencent stopped short of claiming it "comprehensively surpasses GLM 5.2."

For context: GLM 5.2 is currently the strongest model domestically in the coding domain, with virtually no challenger to its position; in general-purpose capability, DeepSeek-V4-Pro occupies a tier of its own. For Hunyuan V3 to push into this league at all is already a meaningful achievement.

Open-Source Deployment: FP8 Quantization Friendly for Local and Enterprise Use

In terms of open-source commitment, Tencent has delivered here — Hunyuan V3 is released not only in BF16 but also with an FP8 quantized version.

On FP8 Quantization: FP8 (8-bit floating point) represents a significant advancement in LLM deployment. Compared to the conventional BF16 (16-bit brain float), FP8 cuts storage per parameter in half, substantially reducing model weight size and VRAM footprint. Modern GPUs such as the NVIDIA H100 and H800 natively support FP8 computation, theoretically enabling roughly 2× throughput gains with minimal precision loss. This allows enterprises and research institutions to deploy the model without requiring top-tier GPU clusters.

In terms of model weight size:

• FP8 version is approximately 300 GB
• Devices with ~400 GB+ of VRAM can run the quantized model
• Note that additional overhead for KV Cache and other components must still be accounted for in pure VRAM deployments

On the deployment framework side, Hunyuan V3 supports both vLLM and SGLang, and can already be deployed via vLLM clusters.

On vLLM and SGLang: vLLM, developed by the UC Berkeley team, is known for its PagedAttention technology, which efficiently manages KV Cache memory fragmentation and significantly improves concurrent throughput — it has become the de facto standard for enterprise-grade LLM deployment. SGLang, also from the Berkeley ecosystem, focuses on structured output and complex reasoning chain scenarios, excelling in Agent workflows and multi-step tasks. Hunyuan V3's support for both frameworks gives developers the flexibility to choose the inference backend best suited to their workload.

MTP-based inference acceleration can be enabled via a dedicated MTP acceleration plugin, making it particularly suitable for large-scale enterprise cluster deployments. Overall, the FP8 quantization approach is quite deployment-friendly for both local and enterprise use cases. If the model's real-world performance holds up to its advertised claims, its VRAM efficiency could earn Hunyuan V3 a meaningful spot in the open-source community.

Two Core Weaknesses: 256K Context and No Multimodal Support

However, Hunyuan V3 has two hard-to-ignore limitations that directly affect its competitiveness in the Agent era.

Context Window of Only 256K

Hunyuan V3's context window is 256K, with a maximum output of approximately 8K. For comparison:

• DeepSeek: 1M (million-scale) context
• Most domestic flagship models: close to 1M context

In today's Agent application scenarios, a 256K context window is noticeably limiting. Whether for coding tasks or agentic workflows, the context fills up after just a few exchanges, triggering automatic compression.

On MoE Context Degradation: To be fair, capability degradation in long-context scenarios is a well-recognized technical challenge for MoE architectures. The fundamental cause is that MoE routing mechanisms are typically trained on shorter sequences; when input sequences grow substantially longer, the Router's dispatch decisions become unstable — some experts get overloaded while others sit idle — ultimately degrading attention quality and information extraction. On top of that, the linear growth of KV Cache with context length is further amplified in MoE models by multi-expert parallelism. This is essentially a universal pain point for all MoE architectures — even DeepSeek hasn't fully solved it. Keeping the context window at a more conservative limit is sometimes the more pragmatic engineering choice.

No Multimodal Input Support

Hunyuan V3 does not support multimodal inputs. Today's standard for domestic flagship models is essentially: either 1M+ long context, or multimodal capability — and usually both. A model that supports neither multimodal input nor offers more than 256K context is at a real disadvantage in production environments.

Summary: Promising Iteration Trajectory, Critical Gaps Remain

All things considered, Hunyuan V3 shows clear improvements over its predecessor, with Tencent claiming it surpasses DeepSeek-V4-Pro in most areas and approaches — but doesn't fully beat — GLM 5.2. The FP8 quantization deployment option is a genuine plus. But the 256K context window and absence of multimodal capabilities leave it in a somewhat disadvantaged position in today's fiercely competitive LLM landscape.

That said, there's reason to be optimistic about its trajectory: Tencent WorkBody is one of the most active Agent clients in China, with a massive volume of real-world usage data — which means the Hunyuan series could iterate at a rapid pace. Here's hoping Tencent moves quickly to close the context and multimodal gaps and deliver a more competitive next-generation release.

Key Takeaways