Tencent Hy4 Preview In-Depth Analysis: 770B MoE Architecture's Agent Capabilities and Real-World Performance

Tencent Hy4 Preview: A Major New Addition to Open-Source Large Models

In late August 2026, Tencent officially released the Hy4 preview model and open-sourced it under the Apache 2.0 license. This move is particularly noteworthy in the current fiercely competitive open-source large model landscape. According to the model card and OpenRouter real-time page (verified on August 31, 2026), Hy4 adopts a Mixture of Experts (MoE) architecture with a total parameter count of 770B (770 billion), but only 49B (49 billion) activated parameters.

The Mixture of Experts architecture dates back to the theoretical framework proposed by Jacobs et al. in 1991, but it wasn't until Google's Switch Transformer paper in 2017 that it was truly introduced to large-scale language models. The core idea of MoE is to split the model's feed-forward network layers into multiple independent "expert" sub-networks, with a gating network selectively activating only a few experts during each inference. This means the model can have an extremely large total parameter count to store extensive knowledge, while only invoking a small subset of parameters when processing each token, achieving a balance between capability and efficiency. In recent years, from Google's Gemini series to Mistral's Mixtral, and DeepSeek-V2/V3, MoE has become the mainstream technical approach for trillion-parameter models.

This "large but sparse" design philosophy represents the mainstream evolution direction of current top-tier large models—storing knowledge through massive parameter capacity while controlling inference costs through sparse activation mechanisms. For teams hoping to deploy powerful models in local or private environments, the permissive Apache 2.0 license means virtually no commercial usage barriers. Apache 2.0 is a permissive open-source license established by the Apache Software Foundation and is one of the most enterprise-friendly licensing methods in the current AI open-source field. Unlike GPL-series licenses, Apache 2.0 does not require derivative works to also be open-sourced (i.e., no "viral" clauses), allowing enterprises to conduct secondary development, fine-tuning, or even commercial closed-source deployment based on open-source models without disclosing their modifications. Additionally, Apache 2.0 includes explicit patent grant clauses, providing users with additional legal protection. In contrast, while Meta's LLaMA series is also open-source, early versions carried commercial usage restrictions that to some extent hindered enterprise adoption.

Technical Architecture: Sparse Attention and Million-Token Context

DeepSeek-Style Sparse Attention Mechanism

Hy4's technical approach draws from industry frontier achievements, adopting a DeepSeek-style sparse attention mechanism. The core value of sparse attention lies in: significantly reducing computational complexity when processing ultra-long sequences, avoiding the quadratic growth overhead of traditional full attention mechanisms as sequence length increases.

Traditional Transformer's self-attention mechanism requires calculating correlation weights between every token in the sequence and all other tokens, with computational complexity of O(n²), where n is the sequence length. When the context window expands to million-token levels, this quadratic growth causes memory and computational requirements to explode. DeepSeek's sparse attention solution in its V2 and V3 series models, through techniques like Multi-head Latent Attention (MLA), dramatically compresses KV cache and allows each token to focus only on the most relevant parts of the sequence rather than all positions. This design reduces attention computation from O(n²) to near O(n·log n) or even better levels, making million-token contexts engineering-feasible. Hy4's adoption of this approach indicates this technical paradigm is moving from single laboratories to become an industry-wide standard.

This design directly supports another key highlight of Hy4: a 1 million (1M) token context window. Million-token context means the model can process entire books, large code repositories, or massive documents in a single pass, which is crucial for code understanding, long document analysis, and complex agent tasks. Specifically, 1 million tokens equate to approximately 7.5 million English words or about 15 million Chinese characters—enough to contain multiple complete novels or all code files from a medium-to-large software project. In practical applications, legal and financial industries can input entire contracts or annual reports for analysis in one go; software engineers can have the model understand an entire codebase's architecture rather than just individual files; RAG (Retrieval-Augmented Generation) systems can directly insert more retrieval results into context, reducing information loss. Previously, Claude 3 and Gemini 1.5 Pro pushed context windows to 200K and 1M token levels respectively, but cases of achieving million-token context in open-source models remain limited. Hy4's emergence is changing this landscape.

MoE Architecture's Efficiency-Capability Balance

With 770B total parameters paired with 49B activated parameters, the activation ratio is approximately 6.4%. This extreme sparsity allows Hy4 to maintain powerful capabilities while keeping actual computational consumption during inference within reasonable bounds. Compared to dense models, MoE architecture can accommodate more "expert knowledge" at the same inference cost—this is the fundamental driving force behind top models' recent shift toward MoE approaches.

Agent Benchmark Tests: Terminal-Bench and DeepSWE Real-World Performance

Performance on Two Core Agent Benchmarks

Hy4's product positioning clearly targets the hottest Agentic (intelligent agent) application scenarios of the moment. Agentic AI is one of the most important evolution directions in the AI industry from 2024-2026. Unlike traditional single-turn Q&A or text generation, agent models can autonomously plan tasks, invoke external tools (APIs, databases, file systems, etc.), adjust execution paths based on intermediate results, and ultimately complete complex multi-step objectives. From OpenAI's Function Calling and Code Interpreter to Anthropic's Computer Use, and various AutoGPT frameworks and Devin-like AI programming assistants, agentic capabilities have become the core competitive focus of leading model vendors. According to predictions from multiple analytical institutions, the AI Agent market will surpass $100 billion by 2027, covering broad areas including automated programming, IT operations, data analysis, customer service, and more.

In two key benchmark tests, Hy4 delivered noteworthy results:

• Terminal-Bench: 85.4 points — measuring the model's ability to execute commands and complete automation tasks in terminal environments
• DeepSWE: 64.3 points — evaluating the model's comprehensive performance in real software engineering tasks

Terminal-Bench is a recently emerged class of agent capability evaluation benchmarks specifically designed to assess large language models' ability to execute practical tasks in terminal (command-line) environments. Test scenarios typically include: writing Shell commands based on natural language instructions, manipulating file systems, managing processes, configuring system services, parsing logs, and troubleshooting failures. Unlike traditional text generation evaluations, Terminal-Bench requires models not only to understand instruction intent but also to generate precise command sequences that can be directly executed, and dynamically adjust strategies based on execution results in multi-step interactions. Hy4's 85.4 score is quite impressive, meaning it can provide correct or near-correct operation solutions in the vast majority of test scenarios, which has direct commercial significance for DevOps automation, IT operations assistance, and similar scenarios.

DeepSWE is a more challenging software engineering evaluation benchmark evolved from SWE-bench. SWE-bench was originally proposed by a Princeton University research team in 2023, requiring models to locate problems in complete code repositories based on real GitHub issue descriptions and generate correct code patches. DeepSWE builds on this foundation by adding more complex dimensions such as multi-file modifications, dependency relationship understanding, and end-to-end test validation. Hy4's 64.3 score indicates it can generate effective solutions for nearly two-thirds of real software engineering tasks—a level already sufficient to play a supporting role in actual continuous integration (CI) processes, helping developers accelerate problem resolution.

Why Agent Capabilities Have Become Core Competitiveness

As large models evolve from "conversational assistants" to "autonomous executors," agentic capabilities have become key indicators for measuring models' practical value. A model capable of independently completing terminal operations, writing and debugging code has commercial value far exceeding simple text generation tools. Hy4's outstanding performance on these two benchmarks indicates Tencent is focused on building a foundational agent model that can truly be deployed in production environments.

Pricing and Market Positioning Analysis

OpenRouter Platform Pricing Details

On the OpenRouter platform, Hy4 is priced at $0.834/million tokens for input and $2.501/million tokens for output. OpenRouter is a third-party AI model routing platform that aggregates API services from OpenAI, Anthropic, Google, Meta, Mistral, and various open-source models, providing developers with a unified interface and flexible model-switching capabilities. Through OpenRouter, developers can quickly compare different models' performance and costs without modifying code.

In the current API pricing ecosystem, the price range spans widely: lightweight open-source models (such as small-scale versions of the Llama 3 series) can have input prices as low as $0.05/million tokens, while top-tier closed-source models like GPT-4o and Claude 3.5 Sonnet typically have input prices in the $2-$5/million token range. Hy4's pricing positions it in the middle ground, achieving a good balance between cost-effectiveness and capability. Considering the 770B parameter scale and million-token context capability, this price offers an attractive option for developers who need long-context processing and strong agent capabilities while still wanting to control costs.

Hy4's Core Competitive Advantages

Hy4's emergence further consolidates Chinese tech giants' voice in the open-source frontier. Its several core advantages can be summarized as:

1. Apache 2.0 commercially-friendly license — significantly lowering enterprise adoption barriers
2. Million-token context + sparse attention — oriented toward long document processing and large codebase analysis
3. Outstanding agent benchmark performance — aligned with intelligent agent application development trends
4. Reasonable API pricing — achieving balance between capability and cost

Summary and Outlook

Tencent's Hy4 preview release marks another step forward for open-source large models in both agentic capabilities and long-context processing dimensions. The 770B total parameter, 49B activated MoE architecture, paired with DeepSeek-style sparse attention, enables it to handle complex tasks while maintaining inference efficiency.

As a "preview version," Hy4's official release may bring further performance optimizations and feature enhancements. However, from current public benchmark data, it already possesses the capability to compete head-to-head with mainstream models in production environments. For teams focused on open-source ecosystems, agent development, and long-text applications, Hy4 is worth including in technical selection evaluations.

It should be noted that all data in this article is based on the model card and OpenRouter real-time page information verified on August 31, 2026. As the model iterates and updates, specific metrics may change.