DeepSeek Open-Sources DeepInfer Inference Engine — Why Did It Trigger a US Ban?

DeepSeek has open-sourced its own inference framework, DeepInfer, which supports commercial use. This underlying engine powers Token generation in high-concurrency scenarios, and while its official performance numbers are impressive, it has also drawn a ban from the US government. Behind this lies a complex game involving open source, performance, and geopolitics.

DeepInfer: An Inference Engine Built for MoE Models

According to a Bilibili video, DeepInfer, the framework DeepSeek^Ad open-sourced this time, is an inference framework designed specifically for high-concurrency scenarios. It optimizes load balancing and speculative decoding for MoE (Mixture of Experts) models and is compatible with NVIDIA and Ascend hardware. Official figures show a single cluster can reach a daily throughput of 155 trillion Tokens, an input cache hit rate of up to 96%, and chat cache retention of up to seven days.

What do these metrics mean? Simply put, the efficiency of an inference engine directly determines the cost of AI services. DeepSeek's earlier ability to run V4.1 Flash at 250 Token/s relied precisely on this engine. Some users have made rough theoretical calculations: if a single card ran at full load all day, daily revenue could exceed 600 billion RMB—of course, this is only a theoretical estimate, and actual revenue depends on runtime and statistical methodology. But it is enough to show the enormous commercial potential of high-performance inference.

MoE (Mixture of Experts) models are a sparse-activation neural network architecture: the model contains multiple "expert" sub-networks, and each forward pass activates only a small subset of them (usually routed through a gating network). This allows the total parameter count to be extremely large (hundreds of billions or even trillions), while the actual computation per inference is far smaller than that of a dense model with the same parameter count.

The advantage of MoE is "large capacity, low compute cost," but the price is load imbalance—some experts may be over-invoked while others sit idle, leading to lower hardware utilization and latency jitter. Therefore, inference engines need to perform expert load-balancing scheduling; at the same time, because of MoE's sparsity, simply increasing compute does not translate directly into throughput—it must be paired with speculative decoding (using a small model to draft multiple Tokens and then having the large model verify them in parallel) to improve generation efficiency. DeepInfer's optimization for these characteristics is precisely the key to its daily throughput of 155 trillion Tokens.

Speculative Decoding is an inference technique that accelerates autoregressive generation in large models. Conventional generation produces only one Token per step, and each step requires running the full large model, resulting in a great deal of serial waiting. Speculative decoding works by first using a smaller, faster "draft model" to continuously predict several candidate Tokens, then having the large model verify these candidates in parallel all at once—Tokens that pass verification are accepted directly, while those that fail are rolled back and recomputed from the first error.

Because the large model's parallel verification overhead is close to a single forward pass, overall throughput can be significantly improved while the output distribution remains consistent with the original model (without quality loss). Under the MoE architecture, differences in expert routing between the draft model and the large model introduce additional challenges, requiring the engine to make targeted adaptations. DeepInfer's combination of speculative decoding with MoE load balancing is an important technical source of its high-throughput figures.

The Industry Significance Behind Open Sourcing: Lowering the Barrier, Promoting Broad Access

DeepSeek's open sourcing and support for commercial use is seen as a key step toward lowering the barrier to high-performance inference. More teams can deploy AI services at lower cost, especially enterprises that need MoE models. The video notes that this is good for the entire industry—open source means technology is no longer monopolized by a few giants, and small and medium-sized developers can also use top-tier inference optimization.

Combined with DeepSeek's consistent open-source strategy, from model weights to inference engine, its roadmap is clear: build an ecosystem through open source, attract developers, and thereby drive technological iteration. This model is not unfamiliar in the AI field, but DeepSeek's execution and performance data have indeed caught the industry's attention.

The US Ban: Security Concerns or Technological Competition?

However, just as the open-source news landed, the US ban arrived. According to the video, the US Department of Homeland Security issued an order prohibiting federal civilian agencies from purchasing or using DeepSeek products, including even derivative services developed with DeepSeek technology. The order affects civilian departments such as the Treasury and the Department of Justice, but excludes national security systems.

What is intriguing is the timeline: the order was issued before July 24 but was kept under wraps until now. The video's author comments sharply on this, calling it political manipulation. From a technical perspective, the reasons for bans usually revolve around data security and supply chain risks, but DeepSeek's open-source nature makes the actual effect of such a ban questionable—the code is already public, and anyone can download and use it.

Are ordinary companies and individuals affected? The document does not explicitly prohibit private use, but the attitude at the government level may influence corporate procurement decisions. For DeepSeek, losing the US federal market is certainly a loss, but the global diffusion capacity of its open-source ecosystem may offset part of the impact.

The US Department of Homeland Security (DHS) is a cabinet-level department of the US federal government, established after the "9/11" attacks in 2001, and is mainly responsible for border security, cybersecurity, critical infrastructure protection, and emergency management. Its subordinate Cybersecurity and Infrastructure Security Agency (CISA) plays an important role in federal procurement security reviews.

The order issued by DHS this time is a federal procurement and use restriction, and the entities it binds are federal civilian departments (such as the Treasury and the Department of Justice), not all US citizens or private enterprises. Such executive orders usually cite reasons such as supply chain security and data leakage risks, but their actual effect is limited by the scope of enforcement and exemption clauses (such as the exclusion of national security systems). Understanding this institutional background helps to judge the actual boundary of the ban's impact on DeepSeek: it is more a "door closing" at the government procurement level than a physical block on the spread of open-source code.

The Game Between Open Source and Bans: The New Normal of AI Geopolitics

This incident reflects the growing geopolitical tension in the AI field. On the one hand, open source is a driver of technological democratization; on the other hand, governments are tightening security reviews of AI technology. The DeepSeek case shows that even if the technology is open source, the national identity behind it will still attract regulatory attention.

At the end, the video's author shouts "open source, open source, and still open source." Behind this emotional expression is a commitment to the ideal of technology without borders. But the reality is that the AI supply chain is being cut apart by politics. For developers, choosing a technology stack may require more consideration of compliance risks; for enterprises, diversified deployment is becoming a trend.

DeepInfer's open sourcing is undoubtedly a display of technical strength, but the accompanying ban also reminds us: AI competition is no longer just about performance metrics, but also a contest of rules and ecosystems.

One-line Summary

DeepSeek open-sourced the MoE inference engine DeepInfer for commercial use, but it was banned by US federal civilian agencies.

Paragraph Summary

DeepSeek has open-sourced DeepInfer, an inference framework designed for high-concurrency scenarios that optimizes load balancing and speculative decoding for MoE models, is compatible with NVIDIA and Ascend hardware, and supports commercial use. Official data shows a single cluster can reach daily throughput of 155 trillion Tokens, an input cache hit rate of 96%, and chat cache retention of up to seven days; its efficiency directly determines AI service costs, and the high performance of V4.1 Flash previously relied on this engine. Open sourcing is seen as a key step toward lowering the barrier to high-performance inference and promoting broad access to technology. However, the US Department of Homeland Security issued an order prohibiting federal civilian agencies from purchasing or using DeepSeek products and derivative services; the order was issued before July 24 but disclosed later. The article argues that the open-source nature makes the ban's actual effect questionable, but the incident reflects the new normal of the game between open source and geopolitics in the AI field.