Liang Wenfeng Open-Sources DeepInfo Inference Framework, Takes On U.S. Ban

After Open-Sourcing Models, Liang Wenfeng Pulls Out an Inference Framework

According to an analysis by Bilibili creator "Bamboo," DeepSeek^Ad has followed up its open-source models by further releasing its inference framework, DeepInfo. What does this mean? Here's an analogy: open-sourcing a model is like handing you a well-trained chef—but no matter how good the chef is, running a restaurant is a different matter entirely. Serving ten tables a day versus a hundred tables flooding in per hour—kitchen scheduling, prep order, running multiple pots in parallel—these are all engineering problems.

AI models work the same way: getting the weights is only the starting point. Making them run fast, cost less, and serve more people simultaneously depends on a complete inference engineering stack. The inference framework is precisely the key component for organizing computation and scheduling tasks. By making its own efficient inference methods public, DeepSeek is essentially handing over the kitchen's SOP as well.

GPU-CPU Cross-Pipelining, Running Full-Tilt Without Waste

You may not have noticed DeepInfo's pipeline design. According to the video, it builds a pipeline where GPU and CPU work in alternation: as one side finishes, the other already has tasks queued up, running full-tilt without wasting a single second.

This heterogeneous scheduling approach isn't new, but building it into an open-source inference framework and adapting it to multiple chips is where the real barrier lies. For teams needing large-scale deployment, what's saved isn't just time—it's real money in compute costs.

Heterogeneous computing scheduling refers to distributing computational workloads appropriately across different hardware architectures—GPU, CPU, and others—within a system, executing them in parallel based on task characteristics. GPUs excel at high-throughput matrix operations, while CPUs excel at logic control and serial tasks. Inference frameworks use pipelines to split preprocessing, model computation, and postprocessing stages, letting CPU and GPU alternate and overlap execution, thereby hiding data transfer and wait times and improving overall utilization. This design requires fine-grained task orchestration and memory management, and is a key engineering barrier in inference frameworks.

Small-Model Prediction + Large-Model Verification, Cutting 30% of Wasted Work

To address slow token generation, DeepInfo also introduces a "confidence prediction" mechanism: a small model guesses answers in advance, and reliable ones are handed to the large model for verification. According to the video, this directly cuts 30% of wasted work—squeezing real gains out of every crack in compute.

This "speculative inference" approach has been explored in the industry, but DeepSeek has engineered it and open-sourced it, lowering the barrier for small and mid-sized teams to build their own efficient inference services.

Speculative inference is an inference optimization technique that accelerates text generation through "small-model drafting + large-model verification." The core idea: use a smaller, faster model (or a lightweight version of the same model) to generate multiple candidate tokens in parallel, then have the large model verify in one pass whether these candidates are acceptable. If the small model's predictions align closely enough with the large model's distribution, the number of forward passes through the large model can be greatly reduced without sacrificing output quality, thereby lowering latency and compute costs. The key lies in the alignment between the draft model and the target model, and the parallel efficiency of the verification stage. Solutions like Medusa and Lookahead already exist in the industry; DeepSeek's engineering and open-sourcing of this technique helps small and mid-sized teams achieve efficient inference at low cost.

Full Chip Adaptation: An Open-Source Counterstrike Under the U.S. Ban

On chip adaptation, according to the video, DeepSeek has fully adapted to both Huawei Ascend and NVIDIA chips. That said, the U.S. has just issued a nationwide ban to blacklist DeepSeek across the entire federal system, even cleaning up third-party services that use its technology—citing the same old data-security rationale.

The creator's view is clear: if you're used to building high walls and collecting tolls, I'll build roads for everyone; if you ban everywhere, I'll open-source everything. What Liang Wenfeng wants is obviously not a few tolls—it's to flatten the barriers to doing AI worldwide, let the whole industry get rich fast, and leave him alone to grind on AGI.

What It Means for the Industry

For business leaders, this wave of open-sourcing is indeed a gift: lower inference costs mean AI applications can be deployed more cheaply, and chip adaptation means more flexible supply-chain choices. But be clear-eyed: an open-source framework is not plug-and-play—engineering deployment still requires teams to digest and tune it.

It should be noted that the technical details about DeepInfo above come primarily from a single video source, and specific performance figures (such as the 30% savings) have yet to be verified by official documentation or more independent testing. Until cross-validated by multiple sources, treat them as directional references rather than precise metrics.

Regardless, open-sourcing models and then open-sourcing an inference framework points to a clear signal: DeepSeek is advancing "affordable AI" from the model layer to the engineering layer.