Why DeepSeek Has US Giants Worried: Breaking Down Four Efficiency Advantages and the Open-Source Moat

DeepSeek leads through four stacked efficiency advantages and thin margins, with open designs that rivals can't replicate.
The article explains why DeepSeek keeps shipping powerful models at extremely low cost: not a single trick, but four stacked efficiency advantages — an asymmetric MoE architecture (8B activated parameters for reading, 16B for generation), extreme KV cache compression, 8-bit/4-bit low-precision storage, and its own inference engine DeepInfer with speculative decoding. Individually modest, together they support prices far below US closed-source models. Open source doesn't mean copyable: cache management and load-balancing code stay private, and DeepInfer supports only specific chips. With deliberately thin margins, users as a byproduct on the road to AGI, and patient backers, DeepSeek's moat is engineering depth and cost structure — not secrecy. Competition is shifting from model design to chips and engineering capability.
One Release, Two Shocks
On September 10, DeepSeekAd released v4.1 Flash — a model with 552 billion parameters that nonetheless outperforms v4 Pro. A little over a month later, on October 20, they open-sourced their inference engine DeepInfer outright. An extremely capable model priced at a fraction of US closed-source models, backed by far less capital investment.
This rhythm keeps repeating: every time DeepSeek releases something, US labs scramble to figure out how it was done. Which raises two questions: how does DeepSeek keep pulling this off? And if the models are fully open-source, why don't US labs just copy them and squeeze DeepSeek out of the market?
The answer isn't some single trick — it's four layers of efficiency advantages stacked on top of each other.
Four Efficiency Levers: Saved Compute Compounds
Lever One: Don't Use the Whole Brain Every Time
DeepSeek's models use a Mixture-of-Experts (MoE) architecture, split into a large number of expert sub-networks, with only a small subset activated per task. V3 has 671 billion parameters, but activates only 37 billion per token. v4.1 Flash goes further: reading input uses just 8 billion activated parameters, while writing answers uses 16 billion.
That 8-and-16 split is deliberate — an asymmetric design built specifically for AI Agents. Agents read tens of thousands of tokens of context each turn but write only a few hundred, so making reading cheaper than generation matches the shape of real workloads. Some analyses note that 552 billion is the main model, with a separate 196 billion-parameter memory module, totaling roughly 748 billion.
The core idea of Mixture-of-Experts (MoE) is to split a massive feed-forward network into multiple parallel "expert" sub-networks and introduce a lightweight routing network. For each input token, the router selects only a few experts to participate in computation while the rest stay silent. This way, the model's total parameter count can reach hundreds of billions or even trillions, but the actual computation per forward pass scales only with the number of activated experts. MoE's advantage is trading larger parameter capacity for stronger expressiveness while keeping inference costs within an acceptable range; the cost is that training is prone to routing imbalance and extreme divergence in expert utilization, requiring additional load-balancing losses to constrain.
Lever Two: Compress Memory to the Extreme
When a model reads long prompts, it keeps notes on everything it has seen — this is the KV cache, the main GPU hog in long-context scenarios. DeepSeek's approach is to compress these notes and then retrieve only the most relevant parts.

At a million-token length, v4.1 Flash requires roughly 10% of the compute of v3.2 and about 7% of the cache memory. And v4.1 Flash cuts the cache further to a quarter of high-speed memory and an eighth of disk storage.
The cache directly determines price because Agents resend a growing context every turn — effectively paying for the same cache over and over. DeepSeek's published cache-hit price is 0.3 cents per million tokens (off-peak). One analyst calculated a coding task reading 10 million tokens (mostly cache hits): about 21 cents on v4.1 Flash versus about $11 on Claude Opus 5 or GPT 5.6 Sol. That's just one blogger's estimate based on list prices, but it's enough to show how the savings compound.
KV cache is a key mechanism in Transformer inference. During autoregressive generation, each new token requires attention computation against the Key and Value vectors of all previous tokens. To avoid recomputation, the inference engine stores the K and V vectors of historical tokens — this is the KV cache. Its memory footprint grows linearly with context length, reaching tens of GB at million-token scales, directly eating into GPU memory and limiting concurrency. Therefore, compressing the KV cache and retaining or retrieving only the most relevant parts is one of the most direct ways to reduce long-context inference costs, and a key optimization focus for inference engines.
Lever Three: Use Fewer Bits
Numbers inside the model can be stored at high or low precision. DeepSeek stores most of the cache in 8-bit format, while the component responsible for "deciding where to look" runs at 4-bit. Lower precision means less memory and faster math.
Low-precision quantization means representing model weights or activations with fewer bits, for example dropping from 16-bit floating point to 8-bit integers or even 4-bit. Fewer bits means more parameters and cache fit in the same memory, less memory bandwidth pressure, and higher matrix multiplication throughput. The difficulty is precision loss: lower-bit representations introduce rounding errors that can degrade output quality. In practice, techniques like per-channel scaling, outlier preservation, and mixed precision are used to mitigate this. DeepSeek's 8-bit cache storage and 4-bit routing computation are essentially a fine-tuned trade-off between precision and cost.
Lever Four: Run the Model Well — DeepInfer
If the model is a recipe, DeepInfer is the kitchen. It uses speculative decoding — cheap draft steps guess several upcoming words first, then the large model verifies them; there's also a prefill shortcut that DeepSeek says nearly halves the compute needed to read prompts. DeepSeek says a single cluster serves 155 trillion tokens per day.
Each layer alone is a modest gain; stacked together, they're what lets a model of this scale be sold at this price.
Speculative decoding is an inference technique that accelerates generation without changing the output distribution. It first uses a smaller, faster draft model (or a lightweight branch of the same model) to guess several candidate tokens in sequence, then has the target large model verify these candidates in parallel in one pass. Accepted tokens are kept directly, and resampling starts after the first rejected position. Since one forward pass of the large model can verify multiple tokens and the draft step itself is cheap, overall generation speed often improves by one to several times. The speedup depends on the draft's acceptance rate: the more accurate the draft, the greater the gain; a poor draft can actually increase overhead.
Open Source Doesn't Mean Copyable: Why Rivals Can't Reproduce the Results

Since the models are open-source, can't big labs just take them and use them? In a sense, they already are. Meta's Llama 4 adopted an MoE design similar to DeepSeek V3; Thinking Machines, founded by former OpenAI CTO Meera Murati, based its first model Inkling primarily on the DeepSeek V3 architecture; GLM5 adopted DeepSeek's latent attention and sparse attention.
The reverse is also true: DeepInfer was initially built mainly on the open-source engine VLLM and drew on SGLang and NVIDIA's TensorRT-LLM, and the VLLM team has publicly thanked DeepSeek. Open source is a cycle, and everyone is in it. DeepSeek borrows from rivals too: v4 was trained with the Muon optimizer (from Moonshot, validated on its Kimi K2), upgraded residual connections follow ByteDance's ideas, and the post-training distillation format credits Thinking Machines in its report.
Among China's open-weight labs, the same components circulate between them within weeks, often with attribution — Tencent's latest model card explicitly notes inspiration from DeepSeek and GLM.
But copying a design isn't copying the result. A paper shows the model's shape but doesn't hand you the GPUs, the years of experience, or the experiments behind every choice. Even DeepInfer is only part of the picture.

DeepSeek did not open-source its cache management and load-balancing code, citing tight coupling with internal infrastructure. DeepInfer supports only one model and runs only on NVIDIA's Blackwell chips and Huawei Ascend, not on the earlier Hopper generation.
Business Model: Why Secrecy Isn't Required to Survive
If the design can be taken, why hasn't DeepSeek been squeezed out? Because its business was never built on secrecy.
First, the hard part isn't the design — it's running it cheaply. Founder Liang Wenfeng has said that even if the model is open-source and everything is shared, the barrier to entry remains extremely high, and running it at low cost is even harder. Part of the advantage lies precisely in the serving layer — which is exactly where the cache management and load-balancing code withheld from DeepInfer lives.
Second, it reportedly keeps margins deliberately thin. In a leaked investor call transcript, Liang reportedly described API pricing as: recouping hardware costs in about 10 months, then stopping there. The logic is simple — set fat margins and rivals can take your open weights and undercut you; thin margins themselves become the moat. DeepSeek has not confirmed the transcript is authentic and has warned users of significant API price increases, so how long this advantage holds remains an open question.
Third, selling to users isn't the main goal. In the same transcript, Liang reportedly called users the sesame seeds on the way to the watermelon, with the watermelon being AGI. Chatbots and APIs are just byproducts of research; DeepSeek isn't bogged down fighting copycats in the product market.
Fourth, it has backers who can afford to wait. DeepSeek spun out of the quantitative hedge fund HighFlyer and has since completed a first external funding round of about $6.9 billion. According to the founder, these backers don't demand quick returns on the chatbot.

After Efficiency Goes Public, Where Does Competition Shift?
DeepSeek keeps disrupting the AI industry, and there's no single secret. It stacks a large number of unglamorous efficiencies: sparse experts, compressed memory, low-precision numbers, a lean inference engine — and then makes most of it public.
Precisely because it's public, the ideas spread. Rivals are already building on DeepSeek's designs, and the advantage shifts from "who has the best design" to "who has the chips and engineering capability to run it cheapest." Once the design is public, the race becomes about who can run it best.
That also explains the anxiety of US giants: they're not facing a rival that wins through secrecy, but one that lays its cards on the table and is still hard to catch up with — because of engineering depth and cost structure.
One-line Summary
DeepSeek stays ahead through four stacked efficiency advantages and a thin-margin strategy — and its open designs are hard to replicate.
Paragraph Summary
The article breaks down why DeepSeek keeps launching powerful models at extremely low cost: it's not a single trick but four stacked efficiency advantages — an asymmetric mixture-of-experts architecture (only 8 billion activated parameters for reading, 16 billion for generation), extreme KV cache compression, 8-bit/4-bit low-precision storage, and its own inference engine DeepInfer (with speculative decoding and a prefill shortcut). Each gain is modest alone; stacked, they support prices far below US closed-source models. The article further notes that open source doesn't mean copyable: key engineering code for cache management and load balancing isn't public, and DeepInfer supports only specific chips. Combined with deliberately thin margins, users as a byproduct on the road to AGI, and backers who can afford to wait, DeepSeek's moat isn't secrecy but engineering depth and cost structure. The competitive focus is shifting from model design to chips and engineering capability.
Related articles

Gemini 4 Neon Leak: A Potential Strong Contender Against Opus 5.5
A Hacker News thread suggests Gemini 4 Neon looks like an Opus 5.5 competitor. This article unpacks the product positioning behind this community signal, shifts in the top-model competitive landscape, and what it means for developers' model selection strategy.

Claude Fable 5.2's Self-Authored Short Film 'Fable' Sparks Discussion
A short film 'Fable,' credited to Claude Fable 5.2 for script, visuals, and directing, appears on Hacker News. This article analyzes the significance of full-pipeline AI creation, community reactions, and what it means for content creators.

Anthropic IPO Pricing: A Signal of AI Giants Going Public
Anthropic's IPO pricing draws attention. This article analyzes its potential impact on AI industry valuation, commercialization paths, and the competitive landscape, and highlights public-market expectations and challenges for AI companies.