Qualcomm CEO's Bold Claim: 100B-Parameter AI Models Could Live Permanently on Smartphones by 2028

Qualcomm Floats an Ambitious Vision: 100B-Parameter Models Resident on Phones

Qualcomm's CEO recently revealed that some AI companies want smartphones to be capable of continuously running 100-billion-parameter large language models by 2028. The statement sparked intense debate in the tech community — it represents both a bold ambition for on-device AI and a stark reminder of the enormous gap that still exists between today's mobile hardware and large-scale models.

The core logic isn't hard to follow: as data center compute costs keep climbing, offloading inference to user devices becomes an appealing way to distribute costs, protect privacy, and reduce latency. But judging from the heated community discussion, there are still some very hard problems to solve before "running a 100B model on your phone" becomes a reality.

How Much Memory Does It Actually Take? A Precise Technical Debate

One of the most valuable threads in the discussion centers on exactly how much RAM a 100B model would require.

One claim suggested that NVFP4 quantization would require only 128GB of phone memory — but it was quickly shot down by more rigorous math. Using the industry-standard estimation formula:

Memory (GB) ≈ bit-width / 8 × parameter count (billions)

At 4-bit quantization, a 100B-parameter model's weights consume roughly 4/8 × 100 = 50GB. Add the KV cache needed for context — modern models with 64k–128k context windows require a few additional gigabytes — and the total lands around 55GB for low-context inference.

Participants also pointed out that NVFP4 is actually closer to 3.75 bits, not a strict 4-bit format, making the "128GB" figure difficult to justify. In other words, model weights alone aren't the biggest obstacle — the real memory killer is the context window and attention mechanism. As one user joked: "Great, so I'll have just enough RAM for a 10,000-token context window."

Context Is the Real Memory Bottleneck

Using half-precision context with full attention causes memory pressure to spike dramatically. The community's practical recommendation is sparse attention, which compresses the KV cache to roughly one-quarter its original size, allowing limited memory to support longer contexts. This reflects a broader trend: on-device AI engineering is shifting its focus from simply compressing weights toward optimizing dynamic memory overhead at inference time.

Sparse attention is an improvement over the standard full attention mechanism in Transformers. Full attention scales quadratically with context length (O(n²)), meaning that as context grows, both KV cache memory and compute blow up rapidly. Sparse attention reduces this to near-linear complexity by having each token attend only to a subset of the sequence — such as a local window, fixed-stride global tokens, or learned important positions — while dramatically shrinking the KV cache that needs to stay in memory. For on-device inference, this compression is especially critical: with the same memory budget, sparse attention can support roughly 4× the context length of full attention, or reduce peak memory to about one-quarter at the same context length. The tradeoff is that tasks requiring long-range dependencies (e.g., very long document Q&A) may miss critical information under sparse patterns, so in practice it is often combined with retrieval-augmented generation (RAG) to compensate for gaps in attention coverage.

Google's Realistic Approach: Mid-Sized Models Make More Sense

Amidst the aggressive "100B always-on" vision, more grounded reference points emerged. Some mentioned that Google may be involved in this space, but what it actually wants to run on phones could be something closer to Gemma 4 31B — a mid-scale model that runs reasonably well within around 32GB of memory.

This framing is telling: for most smartphone use cases, a model that handles search and reasoning well may be entirely sufficient — you don't necessarily need to stuff 100 billion parameters into your pocket. As one user put it: "Do we really need an ultra-powerful model? If it can search and reason, isn't that enough?" The real value of on-device AI lies in responsiveness and privacy, not blindly scaling up parameter counts.

Thermal, Power, and Hardware Constraints Are Real

Beyond memory, heat dissipation and power consumption are unavoidable obstacles. One participant captured the problem with dry humor: "Great idea — I can run inference at 4 tokens per second on my phone while using the heat it generates to cook a pot of beef stew." Slow inference speeds combined with sustained high loads pose serious challenges for thermal management and battery life.

Memory supply chains were also flagged as a concern — some argued that standard RAM cannot keep pace with demand, and that higher-density, lower-cost solutions from manufacturers like CXMT might be needed. One particularly imaginative idea: burning model weights directly into silicon, eliminating the need to store them in RAM at inference time. The catch is that a baked-in model cannot be updated, prompting immediate warnings that this would be "the perfect planned obsolescence" — every model upgrade would mean buying a new phone.

Business Motivations: Technical Roadmap or Stock Price Narrative?

Notably, the community was quite blunt in its commercial reading of Qualcomm's announcement. One commenter cut straight to it: "Just imagine how much the stock price would pop when they announced this."

Forward-looking roadmaps like this often serve a dual purpose — part technical vision, part capital market narrative. But they also reflect a genuine industry judgment: the unlimited scaling of hyperscale data centers is not sustainable in the long run. Distributing compute to edge devices may be the industry's rational response to cost pressure, not just a marketing play.

Hyperscaling refers to the approach taken by major tech companies of continuously expanding GPU clusters and server capacity to linearly increase AI inference and training capabilities. This model has driven the rapid growth of services like ChatGPT over the past few years, but its marginal costs are becoming a growing concern — power supply bottlenecks, cooling infrastructure investment, and chip supply chain constraints are causing diminishing returns for the "just add more compute" approach. "Edge computing" and "on-device inference" are being reconsidered in this context: if user devices can shoulder a meaningful share of inference load, peak pressure on cloud clusters drops significantly, while also avoiding data compliance risks and network round-trip latency. Qualcomm's push for on-device large models is thus grounded in real structural industry needs — and also positions it favorably in the capital markets narrative. After all, in an era of explosive AI compute demand, the story of turning every smartphone into an inference node is one with enormous imaginative upside.

Conclusion: Big Vision, Long Road Ahead

Based on the community's technical analysis, running a 100B model on a phone is not science fiction — the ~50–55GB weight memory requirement at 4-bit quantization is not out of reach given hardware progress over the next few years. The real challenges are concentrated in context memory, thermal management, power consumption, supply chains, and model update mechanisms.

The more realistic near-term form factor will likely be 32GB of memory running a 30B-class model, paired with sparse attention and retrieval augmentation to handle the vast majority of everyday AI tasks. As for "100B parameters permanently resident on a phone" — rather than a firm target for 2028, it's better understood as a collective thought experiment about where the boundaries of on-device AI might ultimately lie.