DeepSeek Open-Sources DeepInfer: How a 72-Hour Context Cache Rewrites AI Inference Costs

DeepSeek has dropped another technical bombshell—but this time the keyword isn't a smarter model, it's cheaper inference.

According to a South China Morning Post tech report (cross-verified by Bilibili tech creators and in-depth YouTube analysis channels), DeepSeek^Ad has released an open-source inference framework called DeepInfer, paired with its latest model DeepSeek V4.1 Flash, claiming it can slash AI inference costs to a level that forces the industry to redo its math. More intriguingly, a senior DeepSeek researcher described the release on social media as "the decisive battle between light and darkness"—why would an update to a caching tool be elevated to the level of humanity's fate?

What DeepInfer Actually Does: Removing "Repeated Labor" from Inference

To understand DeepInfer's value, we first need to return to a waste the industry has long tolerated: most AI inference recomputes the same things over and over.

In the traditional model, a user uploads a financial report for the model to analyze, and the GPU computes attention weights word by word starting from the very first character. An hour later, another user asks a similar question about the same report, and the system discards all the previous mathematical paths and starts over from scratch. Compute, electricity, hardware wear—everything doubles.

DeepInfer's core mechanism is context caching: it retains already-computed mathematical paths in memory, with a cache validity of over 72 hours. When similar requests arrive, the system directly calls the precomputed intermediate results, skipping the most energy-intensive prefill stage.

Using the metaphor repeatedly cited in podcasts: it's like a restaurant's prep cook chopping ten pounds of onions on Monday and putting them in the fridge, then grabbing them directly when making soup on Wednesday instead of chopping them all over again. What DeepInfer remembers isn't the final answer—it's all the upfront "prep work."

Prefill is the first stage of large language model inference: the model reads the user's entire prompt in one pass, computes Key/Value vectors for each token and stores them in VRAM, establishing the attention context that subsequent token-by-token generation depends on. Compute consumption in this stage is roughly proportional to input length; in scenarios like long-document analysis and long conversation histories, prefill often accounts for the majority of a single request's total time and energy. The contrasting decode stage autoregressively generates output token by token, reading the accumulated KV cache each time. The so-called "repeated computation" refers precisely to the same or similar prefixes being repeatedly prefilled across different requests—the value of context caching is built on persisting this stage's intermediate results so that subsequent requests can reuse them directly.

KV cache (Key-Value Cache) is the technical foundation for understanding context cache reuse. In Transformer self-attention computation, each token produces a set of Key and Value vectors; when generating a new token, the model needs to perform attention operations with the K and V of all previous tokens. To avoid recomputation, inference systems store these vectors in VRAM—the KV cache. Context caching is essentially retaining the KV cache blocks corresponding to a given prefix across requests, and determining whether a new request can hit via prefix matching. The engineering challenges lie in limited VRAM capacity, the need to manage and evict cache blocks at token granularity, and ensuring positional encoding and batch scheduling remain consistent after a hit—otherwise reuse can introduce errors or slow throughput.

Hardware-Agnostic: NVIDIA and Huawei Ascend Run on the Same Efficiency

Another underrated design of DeepInfer is its neutrality at the hardware layer. According to the multi-source analysis above, the framework supports both NVIDIA's SM100 Blackwell architecture and China's Huawei Ascend 950 chips, so enterprises don't need to swap hardware stacks for framework migration.

The strategic significance here outweighs the technical significance. Against the backdrop of continuously tightening US chip export controls, an inference framework that runs indiscriminately across both Chinese and American hardware ecosystems effectively reduces the pressure on global enterprises to pick sides. For buyers, which chip they purchase no longer determines whether they can access efficient inference.

The Economics: $38 per GPU per Hour, Profitable Even in Off-Peak Hours

Technical data released by DeepSeek shows that under a single-server configuration, each GPU can generate about $38 in revenue per hour—and this figure still holds during off-peak hours.

This data point is the most impactful part of the entire release. The economic model of traditional inference servers relies on daytime peaks to amortize nighttime idle costs—when machines sit idle, electricity and cooling bills still come due. DeepInfer pushes the marginal cost of a single inference low enough that even a small number of nighttime requests can cover fixed expenses. For compute renters and small and mid-sized cloud providers, this means the break-even point has been completely rewritten.

DeepSeek team lead Kui Tianyi publicly stated that the company actively wants third-party enterprises to make money with these models. According to analysis, this isn't charity but an ecosystem positioning strategy: once enough startups build their businesses on DeepSeek's open-source stack, it becomes a foundational standard the entire industry cannot easily replace.

Open Source vs. Closed Source: A Route Debate Dressed Up as a Technical Update

What really pushed this release into the spotlight was a long post on social media by senior DeepSeek researcher Chen Deli. He framed it as "the decisive battle of two futures," using "light and darkness" to describe the opposition between open-source and closed-source AI, and said the open-source community will fight until "total victory."

The wording is fierce, but the logic can be broken down: closed-source models mean a handful of corporate executives hold pricing power over the knowledge gateway, access control, and the power to define content boundaries—what can be said and what counts as correct is determined by one company's values. Chen Deli's position is that concentrating this power in a small group is the real risk.

The opposing argument is equally serious. Anthropic CEO Dario Amodei argues that open weights amount to distributing near-weapons-grade capabilities without门槛—er, without barriers: the threshold for writing complex viruses, large-scale disinformation, and even bioweapon design would all be lowered. He has also publicly accused some Chinese companies of building systems on top of American AI technology. Safety and openness—two priorities collide head-on here.

Open weights and open source are often used interchangeably in AI, but they mean different things. Open weights means the parameter files of a trained model can be downloaded and deployed locally; users can fine-tune, quantize, and distill them, but may not receive the training data, training code, or full licensing freedom. A strictly open-source model also publishes its training methods and data sources and permits free redistribution and commercial use. Dario Amodei's concern mainly targets open weights: once the parameters of a high-capability model leave the provider's control, abusers can locally bypass API-level safety filters and usage monitoring. This also explains why "how open" often determines regulatory attitudes and risk boundaries more than "whether open."

Beijing's Dilemma: Export Controls and Open-Source Promotion in Parallel

The Chinese regulatory posture presents a surface-level contradiction. According to reports, Beijing recently proposed draft export controls on 23 AI technologies, but an official spokesperson simultaneously made clear that China will continue to encourage open-source AI development, specifically naming the "Global South"—the developing economies of Africa, Latin America, and Southeast Asia.

The essence of this combination is: control the outflow of core technologies while using free open-source tools to capture the technology foundation of emerging markets.

If this strategy works, independent developers in Nairobi or São Paulo will have inference capabilities on par with Silicon Valley teams, yet face entirely different local problems. The geographic distribution of the next wave of technological innovation may, for the first time, not be determined by California or Beijing.

Export controls in the AI context generally refer to two types of measures: first, export restrictions on hardware such as advanced chips and manufacturing equipment; second, controls on intangible technology transfer such as model weights, algorithms, and technical documentation. The latter is significantly harder to enforce, because weight files can be transmitted over the internet, and once an open-source release happens it is difficult to retract. This also creates an inherent policy tension: encouraging open source benefits ecosystem expansion and international influence, but open source itself conflicts with the goal of "preventing the outflow of core technologies." In practice, regulators often seek compromises through licensing agreements, filing systems, and differentiated treatment of models above certain capability thresholds, rather than outright bans.

Editorial View

DeepInfer's technical core—long-duration context caching—is not an entirely new concept; both Anthropic and Google have similar mechanisms. But DeepSeek made it open source, hardware-agnostic, and backed by verifiable revenue data—the commercial impact of this combination far exceeds any single technical breakthrough. The question truly worth tracking isn't how much electricity caching can save, but where AI competition's next moat will appear once inference cost is no longer a barrier.