DeepSeek V4 Pro In-Depth Analysis: How Open-Source Weights Are Disrupting the Closed-Source LLM Landscape

DeepSeek V4 Pro Official Release: A New Benchmark for Open-Source LLMs

DeepSeek V4 Pro (0813 version, distinct from the previous Preview version) has officially launched, once again pushing the capability boundaries of open-source large language models forward. According to evaluations by the 2 Minute Papers channel, this version demonstrates significant advantages over the smaller Flash version across multiple tasks—especially in scenarios requiring spatial understanding.

In 3D modeling tests, the Flash version failed to fully comprehend the three-dimensional structure of objects, producing results with numerous missing parts and dark areas. The Pro version, however, shows much better structural understanding, generating more complete and coherent results. The improvements in model scale and training strategy are translating into tangible quality leaps.

More notably, DeepSeek has released the model weights under the MIT license, allowing anyone to run the exact same model at a price they can accept, no longer locked into a single vendor's pricing structure.

MIT License and the Legal Framework of Open-Source Models

The MIT license is one of the most permissive open-source licenses in software, allowing anyone to freely use, copy, modify, merge, publish, distribute, sublicense, or sell copies of the software, with the only requirement being retention of the original copyright notice. In the AI model context, adopting the MIT license means enterprises can integrate the model into commercial products without open-sourcing their own code—a stark contrast to "copyleft" licenses like GPL. DeepSeek's choice of the MIT license effectively minimizes barriers to commercial adoption and encourages rapid ecosystem expansion. This strategy represents a distinct market positioning compared to Meta's LLaMA series (which uses custom licenses restricting commercial use) and OpenAI's fully closed approach.

The Core Value of Open Weights: Breaking the Closed-Source Monopoly

The core value of open-source lies in breaking monopolies. When weights are fully open, numerous hosting service providers emerge in the market, competing on price and pushing costs toward reasonable levels. This contrasts sharply with the closed-source model landscape of "one vendor sets the price, no alternatives."

Of course, barriers exist in reality. As the reviewer candidly admits: "I'd love to use it, but I don't have that kind of hardware." Running a model of this scale locally at home is nearly impossible for ordinary users.

The GPU Cloud Platform Market Landscape

The rise of GPU cloud platforms like Lambda Labs stems from the massive demand for high-performance computing resources in deep learning training and inference. Compared to traditional cloud providers like AWS and Google Cloud, specialized GPU cloud platforms typically offer more cost-effective on-demand rental options, primarily targeting AI researchers and startups. A server equipped with 8 NVIDIA H100 GPUs costs approximately $30-50 per hour to rent, which is the minimum configuration needed to run models with hundreds of billions of parameters. Platforms like Lambda, Paperspace, and RunPod compress costs to 60-70% of traditional cloud providers through bulk GPU procurement and optimized scheduling efficiency. However, even so, the monthly cost of continuously running large models can still reach thousands of dollars for individual developers—which is why "running locally at home" is nearly impossible for ordinary users. A single H100 GPU costs over $30,000 on the market, and running top-tier open-source models typically requires multiple cards working in coordination.

Alternative paths include using GPU cloud platforms like Lambda, or directly calling DeepSeek's official hosted service—though the latter recently raised prices significantly, reaching 2.5 to 5 times the previous rates.

Even so, the significance of open weights remains undiminished. No one gets quietly "downgraded" to an inferior model for triggering某个 keyword. You're running exactly the model you chose—transparent, controllable, and reproducible.

Capability Leap Under the Same Architecture: Post-Training Is Key

V4 Pro shares the same model architecture as the Preview version from less than four months ago, yet it's significantly more powerful. The "magic" behind this primarily occurs in the post-training phase after pre-training.

The Core Role of the Post-Training Phase

LLM development consists of three main stages: pre-training, post-training, and deployment optimization. Pre-training involves unsupervised learning on massive text datasets, allowing the model to grasp fundamental language patterns—this phase consumes the vast majority of compute resources (typically over 90% of total costs). Post-training includes techniques like supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), aiming to transform the model from "can talk" to "can do things"—understanding instructions, rejecting harmful requests, and generating content aligned with human preferences. OpenAI's research shows that GPT-3 and InstructGPT use the same pre-trained model, but post-training alone made the latter's practicality increase several-fold. DeepSeek V4 Pro's performance leap while sharing architecture with the Preview version exemplifies post-training technology advancement: specialized expert training reinforces specific capabilities, multi-teacher distillation integrates these capabilities, and multi-token prediction optimizes inference efficiency—the three working together produce a "1+1+1>3" effect.

Multiple Specialized Expert Models Trained Independently

DeepSeek trained multiple specialized models focused on different areas such as mathematics, programming, and Agent tasks. An important clarification: these "experts" are not the same as the "experts" in Mixture of Experts (MoE) architecture.

How Mixture of Experts (MoE) Architecture Works

Mixture of Experts (MoE) is a neural network architecture design pattern dating back to a 1991 paper, recently regaining popularity through Google's Switch Transformer and Mistral AI's applications. The core idea is to decompose a large neural network into multiple "expert" subnetworks, with a "gating network" dynamically deciding which experts to activate for each input. For example, when processing math problems, experts A and C might activate; when processing code problems, experts B and D activate. This sparse activation mechanism allows the total parameter count to be very large (e.g., hundreds of billions) while only using a small portion during each inference, dramatically reducing computational costs while maintaining performance. DeepSeek-V2 is a representative work of MoE architecture, but the "expert models" mentioned in the text refer to completely independently trained model checkpoints—these are concepts at two different levels.

MoE experts are small modules within the same neural network, while the experts here are independently trained complete model checkpoints.

Knowledge Distillation: Ten Teachers Training One Student

Next comes the knowledge distillation phase. DeepSeek uses over ten "expert teachers" to train a final student model, allowing it to absorb capabilities from all sides.

The Evolution History of Knowledge Distillation Technology

Knowledge Distillation was systematically proposed by Geoffrey Hinton and colleagues in 2015, with the core idea being that a small model (student) learns the behavioral patterns of a large model (teacher) rather than directly learning from raw data. The technical key is that the student must not only match the teacher's final prediction results (hard labels) but also learn the "dark knowledge" embedded in the teacher's output probability distributions (soft labels)—for example, if the teacher considers an incorrect answer to have a 5% probability, this subtle uncertainty information is crucial for the student model's generalization ability. Traditional distillation uses a single teacher, while DeepSeek's multi-teacher distillation is a cutting-edge direction in recent years, similar to having a student simultaneously learn from math, physics, and chemistry teachers, with each teacher providing optimal guidance in their respective domains, ultimately forming well-rounded comprehensive capabilities.

During training, the student compares "what I would originally do" with "what the teacher would do," then continuously adjusts outputs to bring them closer to the teacher. Training one student with ten teachers has resulted in significant capability improvements.

Multi-Token Prediction: 78% Generation Speed Improvement

V4 Pro no longer predicts just one token at a time but generates multiple tokens in advance, and does this technical approach better than ever before.

The Technical Breakthrough of Multi-Token Prediction

Traditional autoregressive language models (like the GPT series) use token-by-token generation: predict the first word, then predict the second word based on already-generated content, and so on. This serialized process cannot be parallelized, becoming the main bottleneck for inference speed. Multi-token Prediction attempts to break this limitation, allowing the model to predict multiple future tokens at once. Meta's 2024 research shows that training the model to simultaneously predict the next 4 tokens not only accelerates inference but also improves the model's long-range planning abilities and code generation quality. DeepSeek V4 Pro's 78% speed improvement means that while maintaining output quality, it transforms "think while speaking" into "think first, then say several sentences"—revolutionary for real-time interactive applications (like code completion and dialogue systems). The technical challenge lies in designing training objectives and decoding strategies so that advance-predicted tokens are both accurate and coherent.

DeepSeek claims V4 Pro's generation speed improves by up to 78%—an acceleration users can immediately feel in real-world usage.

From Paper to Production in Just Six Weeks: The Acceleration of Open Innovation

Perhaps most striking is the speed from research to deployment. The related research paper was published just six weeks ago, and now it's already being used by everyone—a top-tier paper landed in everyone's hands for free in just six weeks.

This iteration pace validates a perspective: cutting-edge experiments can rapidly deliver better results, and open sharing benefits the entire ecosystem. Openness isn't charity—it's a more efficient innovation paradigm.

V4 Pro also comes equipped with a newly designed Agent Harness (intelligent agent framework), further expanding the model's capability boundaries in autonomous task execution.

Technical Background of the Agent Harness Framework

Agent is a key concept in AI, referring to systems that can perceive their environment, make autonomous decisions, and execute actions to achieve goals. In the LLM field, Agents typically refer to enhanced LLM systems capable of using tools (calling APIs, executing code, querying databases), performing multi-step reasoning, and autonomously completing complex tasks. The popularity of frameworks like LangChain and AutoGPT has driven explosive growth in Agent applications. Agent Harness is an execution environment specifically designed by DeepSeek for V4 Pro, likely including tool invocation interfaces, task planning modules, memory management systems, and security sandboxes. Unlike general frameworks, a Harness optimized for a specific model can better leverage the model's native capabilities—for example, if the model has seen specific tool invocation formats during pre-training, a specialized framework can directly utilize this built-in knowledge without additional fine-tuning. This deep integration is becoming a new battleground in the competition between closed-source and open-source models.

Conclusion: Open-Source Is Systematically Eroding Closed-Source Moats

The release of DeepSeek V4 Pro demonstrates once again that open weights combined with transparent training methods are systematically eroding the moats of closed-source models. With the same architecture, through more refined post-training (specialized experts + distillation + multi-token prediction), a qualitative transformation can be achieved within four months—showcasing the compounding power of open research.

Whether you use it through official services, third-party hosting, or self-built GPU environments, the open path that DeepSeek represents deserves serious study and participation from every AI practitioner.

Key Takeaways

• DeepSeek V4 Pro achieves significant capability improvements over the Preview version through specialized post-training techniques while using the same base architecture
• Released under the MIT license, enabling full commercial freedom and breaking vendor lock-in
• Three core technical innovations: specialized expert training, multi-teacher knowledge distillation, and multi-token prediction (78% speed boost)
• Demonstrates that open-source models can match or exceed closed-source capabilities through transparent, reproducible methods
• Rapid iteration cycle (research paper to production deployment in 6 weeks) showcases the efficiency advantages of open innovation