DeepSeek V4 Deep Dive: Three Major Innovations — Hybrid Attention, Manifold-Constrained Hyperconnections, and MOM Optimizer Fully Explained

DeepSeek V4 Overview: Reclaiming the Open-Source Throne

DeepSeek V4 has officially launched — the first major version update since DeepSeek R1. Two models were released: DeepSeek V4 Flash, optimized for extreme cost-efficiency, and DeepSeek V4 Pro, the performance flagship. Both models feature a fully upgraded 1-million-token context window. The Pro model matches top closed-source models like Claude Opus 4.6, Gemini 3.1 Pro, and GPT 5.4 across core metrics including reasoning, coding, and Agent tasks, while maintaining a commanding lead among open-source models — making it arguably the definitive open-source LLM of its generation.

More significantly, DeepSeek V4 introduces three fundamental architectural innovations: Hybrid Attention Architecture, Manifold-Constrained Hyperconnections, and the MOM Optimizer, fundamentally redefining the model's capability boundaries from the ground up.

Performance Benchmarks: Going Toe-to-Toe with Top Closed-Source Models

Official Benchmark Results

According to official evaluation data, DeepSeek V4 Pro demonstrates top-tier performance across multiple dimensions:

• Coding: Achieved a high score of 3206 on CodeForces and a 95.2% pass rate on MMT-266
• Agent Tasks: Scored 80.6 on SWE-Verify and 83.4 on BrowsCamp
• Overall Performance: Matches or exceeds the average performance of GPT-5.4 and Gemini 3.1 Pro

However, in pure knowledge Q&A, DeepSeek V4 still trails top closed-source models. On SimpleQA — a benchmark heavily reliant on parametric memory — Gemini 3.1 Pro leads with 75.6 while DeepSeek V4 Pro scores 57.9. That said, compared horizontally within the open-source camp, DeepSeek V4 comprehensively outperforms Qwen 3.5, Kimi K2.6, and Gemma 5.1 across all metrics.

Coding Capabilities in Practice: Full Claude Code Compatibility

DeepSeek V4's API now fully supports Anthropic endpoints, achieving complete compatibility with Claude Code. Anthropic has defined a standardized API specification for its Claude model series, including message formats, Tool Use protocols, and streaming output formats. MCP (Model Context Protocol) is an open protocol proposed by Anthropic to standardize interactions between AI models and external tools/data sources — essentially a "USB standard" for AI. DeepSeek V4's full Anthropic endpoint compatibility means developers can swap in DeepSeek V4 as the underlying model in tools like Claude Code and Cursor without modifying any code, while retaining all advanced features including MCP tool calls and long-term memory. In testing, simply entering DeepSeek's official Anthropic endpoint in the configuration allows direct use of DeepSeek V4 as the base model within Claude Code.

After integrating DeepSeek V4 into the team's proprietary Claude Code external endpoint project, all features — MCP, Skills, Memories, Agile Teams — ran smoothly with perfect compatibility.

Test Highlight: With Claude Code Agile Teams tools connected, DeepSeek V4 Pro built a complete Kapashi AM Wiki knowledge base retrieval system in just 5 minutes — not only displaying relationships between knowledge base documents fluently, but also handling document exchange, smooth Q&A, and deep-diving into knowledge details.

Agent Performance: On Par with Opus 4.6

After integrating DeepSeek V4 into the Fufan Open Cloud system, the experience is virtually indistinguishable from Opus 4.6 — whether it's multi-step tool calling during Agent runtime, loading and running extensions like Skills and MCP, maintaining and optimizing long-term memory, or using Computer Use to operate a local machine. DeepSeek V4 Pro's Agent performance has reached the absolute global top tier.

Cost-Efficiency and Deployment Options

DeepSeek V4 maintains top-tier performance while offering highly competitive API pricing:

Metric	DeepSeek V4 Pro	Claude Opus 4.6
Input Price	¥12/million tokens	~¥36/million tokens
Output Price	¥24/million tokens	~¥168/million tokens

Input costs are roughly one-third of Opus 4.6, and output costs are less than one-seventh — a stark cost-efficiency advantage.

For private deployment, the MoE architecture and underlying innovations deliver significant efficiency gains. Mixture of Experts (MoE) is a core design paradigm for modern ultra-large-scale language models: the feed-forward network layers are replaced with multiple "expert" sub-networks, and during each inference pass, a gating router dynamically selects only a few experts to activate rather than engaging all parameters. This allows the model to maintain an enormous total parameter count (expanding knowledge capacity) while only activating a small fraction during inference (controlling compute costs). DeepSeek V4 Pro's design — 1.6 trillion total parameters with only 49B activated per inference — is a textbook MoE implementation. It theoretically possesses the knowledge breadth of a dense model while incurring inference costs closer to a 49B small model:

• V4 Pro: 1.6 trillion total parameters, 49B activated per inference, ~800GB VRAM required, runnable on dual-node 8×A100/A800
• V4 Flash: 284B total parameters, 13B activated per inference, ~150GB VRAM required, runnable on 2×A800 or 8×4090

Additionally, DeepSeek V4 supports freely switchable hybrid inference modes and is fully compatible with domestic Huawei Ascend series chips, completely eliminating dependency on NVIDIA GPUs. Huawei's Ascend series AI chips represent the most mature large-scale AI training and inference hardware platform in China, with its software stack CANN (Compute Architecture for Neural Networks) positioned as a counterpart to NVIDIA's CUDA ecosystem. As U.S. chip export controls on China continue to tighten — severely restricting access to high-end NVIDIA GPUs like H100 and A100 — DeepSeek V4's full Ascend compatibility is not only strategically significant for its own supply chain security, but also provides the entire domestic LLM ecosystem with a validated technical pathway for "de-NVIDIA dependency." This represents a major milestone in China's domestic LLM development.

Deep Dive into Three Core Technical Innovations

Innovation 1: Hybrid Attention Architecture — The Cost-Cutting Key to Million-Token Context

The standard Transformer's self-attention mechanism has O(n²) computational complexity — meaning if the sequence length doubles, compute and VRAM consumption quadruple. Extending the context window from 32K to 1 million tokens without any optimization would require approximately a 1000× increase in VRAM — completely infeasible in engineering terms. DeepSeek V4 addresses this with a hybrid attention mechanism featuring three parallel pipelines, each serving a distinct purpose:

1. CSA (Compressed Sparse Attention): Acts like an intelligent funnel, lightly compressing historical information at a 4:1 ratio, then using a flash indexer to precisely select the most valuable portions