DeepX Claims Running 100-Billion-Parameter LLMs at 5W
DeepX Claims Running 100-Billion-Param…
DeepX claims running 100B-parameter models within 5W, but operating conditions and details remain unverified
DeepX claims inference on models up to 100 billion parameters under 5W, a figure notable enough in on-device AI, though community response has been muted. The article notes that 5W and 100B are two independent boundary conditions, and the original text doesn't state whether they hold simultaneously under the same operating conditions. On-device large model inference faces three hard constraints: memory bandwidth, energy efficiency, and quantization/sparsification—going from billion-level to hundred-billion-level is not linear extrapolation. If the claim holds, it would affect privacy-sensitive scenarios, offline availability, and cloud-edge compute division, but credibility depends on third-party reproduction, standard benchmark performance, and mass production timelines. Until verifiable results emerge, it should be treated as a direction to be verified.
DeepX recently made a striking claim: running inference on models with up to 100 billion parameters within a power budget of under 5W. After the news appeared on Hacker News, discussion wasn't particularly heated—only 7 points and 1 comment—but the number itself is enough to make anyone working in on-device AI pause and think: if it holds up, what does it mean?
The Ambition Behind a Single Power Number
Let's first put the magnitude in perspective. 5W is roughly equivalent to the whole-device power consumption of a smartphone under moderate load, or the thermal design headroom of a low-power IoT chip. Squeezing inference of a 100B-parameter model into that budget represents a considerable gap from current mainstream practice—cloud inference relies on accelerators in the hundreds-of-watts range, while what runs on-device is mostly small models below the billion-parameter scale.
If DeepX's claim holds up, it points to a fundamentally different hardware approach: rather than compressing large models to the extreme and then adapting them to general-purpose chips, it means designing chip architecture specifically for low-power inference. This kind of thinking isn't new in the on-device AI chip space, but pushing the parameter scale to the tens-of-billions level is the boldest part of the claim. Note that "under 5W" and "up to 100B" are two independent boundary conditions, and the original text doesn't state whether they hold simultaneously under the same operating conditions—whether it's a 100B model running within 5W, or these two limits being reached separately under different configurations. This distinction is critical and currently cannot be confirmed from the limited information available.
The Real Constraints of On-Device Inference
To judge the credibility of this claim, we need to return to several hard constraints of on-device inference.
Memory bandwidth. Large model inference is a classic memory-intensive task—each token generated requires moving a large number of weights from storage to compute units. As parameter scale increases, bandwidth demands inflate rapidly, while the bandwidth budget of low-power chips is typically very tight.
Energy efficiency. Power constraints ultimately come down to how much useful computation can be done per watt. The efficiency gains seen in low-power NPUs in recent years have indeed been significant, but going from the billion-parameter level to the hundred-billion level is not a simple linear extrapolation.
Quantization and sparsification. Compressing a 100B-parameter model into hardware with small memory and low bandwidth almost inevitably relies on aggressive quantization or sparsification techniques. The impact of these techniques on output quality is an unavoidable question when evaluating such claims.
DeepX doesn't provide details on any of these dimensions in this brief claim, which is one reason the community response has been muted—an isolated combination of power consumption and parameter count lacks verifiable context.
MoE architecture. The original text mentions the distinction between dense models and MoE among the questions to be verified, and this deserves elaboration. MoE (Mixture of Experts) splits a model into multiple expert sub-networks, activating only a small fraction of parameters for each token. This way, the model's total parameter count can be large, but the number of parameters actually involved in computation per inference is much smaller—100B total parameters might have only a dozen billion or fewer activated parameters. For power- and bandwidth-constrained on-device hardware, this means the "parameter count" number itself doesn't directly correspond to computational load. If the claimed 100B refers to MoE total parameters, its actual inference cost is on a completely different order from a dense 100B model, and the credibility assessment criteria for the two are also different. This is precisely why asking about model structure is more meaningful than just looking at parameter scale.
Quantization precision. Quantization is the technique of compressing model weights and activations from high-precision floating point (such as FP16) to lower-precision representations (such as INT8, INT4, or even lower), and is a standard approach for deploying large models on-device. Lower precision means smaller memory footprint and bandwidth requirements for the model, but also greater risk of output quality degradation. The industry-standard 4-bit quantization can already maintain good results on large models, while more aggressive 2-bit and below typically requires specialized quantization algorithms or mixed-precision strategies. When discussing "running 100B at 5W," the quantization bit-width directly determines memory usage and bandwidth pressure—it's one of the key variables for judging the feasibility of the claim, and such information is typically not disclosed in brief announcements.
Sparsification. Sparsification refers to making a significant proportion of elements in model weights or activations zero, thereby skipping invalid computation and reducing power consumption. It's divided into unstructured sparsity (zero elements randomly distributed) and structured sparsity (distributed according to rules such as channels or blocks), with the latter being more hardware-friendly because general-purpose or specialized accelerators can more easily skip entire blocks of computation. Sparsification and quantization are often used together to accommodate larger models within limited memory and bandwidth budgets. But the cost is that specialized training or pruning processes may be needed to maintain accuracy, and actual speedup depends heavily on whether the hardware natively supports sparse computation—this should be factored into any evaluation of "running large models at low power" claims.
Potential Impact on the On-Device AI Landscape
Assuming this claim holds under reasonable conditions, the impact would manifest at several levels.
The most direct is privacy-sensitive scenarios: running hundred-billion-parameter models without data leaving the device would redefine the experience boundaries of applications like local assistants and local document processing. Next is offline availability—in environments with unstable or completely absent network connectivity, devices could still provide intelligence approaching cloud-level. Looking further ahead, if low-power large model inference becomes reality, the division of compute between cloud and edge would be renegotiated—which tasks must go to the cloud and which can stay local would have different answers.
Conversely, such claims also need to be viewed within the industry's usual rhythm: peak figures that chip vendors produce before launch using specific configurations, specific models, and specific precision often differ from what developers actually experience after getting their hands on the hardware. What's truly convincing is third-party reproduction on standard benchmarks and stable performance on mass-produced hardware.
The Gap Between Peak Figures and Real-World Performance. Performance numbers published by the chip industry before launch are typically based on carefully selected configurations: specific models, specific input lengths, specific precision, and even specific power measurement methodologies (for example, counting only chip power rather than whole-device power). What developers face after receiving hardware is often more complex real-world workloads—longer contexts, concurrent tasks, thermally constrained sustained operation. Therefore, when the industry evaluates such claims, it places more weight on third-party reproduction on standard benchmarks and sustained performance on mass-produced devices, rather than peak metrics from launch events. This convention is the context for understanding the credibility of this claim.
Questions to Be Verified
Around this claim, several questions deserve continued attention: what specific model configuration and precision correspond to the 5W power consumption; whether the 100B parameters refer to a dense model or a sparse structure like MoE; what the inference throughput and latency performance are; and most critically—the mass production timeline and the form in which developers can access it.
DeepX is not the first company to claim running large models at extremely low power—the on-device AI chip space has seen many players crowd in recent years, and the gap between claims and delivery is also the norm. This number is worth noting down, but until reproducible results are available, treating it as a direction to be verified rather than an established fact is the more prudent attitude.
Related articles

Zed Editor Releases v1.29.2: Observations on a Routine Version Iteration
Zed editor releases patch v1.29.2, version bump by @JosephTLyons, commit verified via GitHub signature. This article covers the release's basic info, what a patch version means, and practical upgrade advice for users.

Anthropic IPO Questioned: Hacker News Debates Timing of Going Public
A Hacker News post titled "Just say no to Anthropic's IPO" sparks debate on the timing of Anthropic's public listing and AI capitalization, touching on valuation bubbles and mission independence.

Hands-On with the Genie 4 World Model: Directing Worlds in Real Time Like a Puppeteer
Google DeepMind's Genie 4 world model in depth: Special Guest X's hands-on demo of real-time scene direction via prompts, custom interactive apps in Project Genie, and prompts-as-seeds reproducibility.