PrismML Breakthrough: How a 27-Billion-Parameter LLM Fits Inside an iPhone

A Bold Attempt to Rewrite the Economics of Mobile AI

In the race to run large language models (LLMs) on mobile devices, a startup called PrismML has put forward a striking result: it has successfully compressed Alibaba's open-source Qwen 3.6 model and run it on an iPhone 17 Pro. The model packs 27 billion parameters — the billions of weighted values in a neural network that are continuously tuned during training on massive datasets, determining how complex the model's data processing can be and how richly it encodes knowledge. More parameters theoretically means richer knowledge encoding and more sophisticated reasoning chains, but storage and compute requirements grow exponentially.

For context, mainstream on-device models that currently run on smartphones typically have only a few billion parameters active at any given time, while the top-tier models with trillions of parameters cannot be deployed on mobile devices at all. The fundamental tension in on-device deployment is this: flagship smartphones typically have 8–16 GB of memory, while uncompressed large models can easily weigh tens or even hundreds of gigabytes. If PrismML's claims hold up, running a fully-activated 27-billion-parameter model on a phone would represent a significant leap in mobile AI capability.

PrismML was incubated at Caltech and is backed by prominent venture firm Khosla Ventures. According to the company, the compressed model can handle complex conversations, logical reasoning, fully autonomous agents, and code generation — and will be released as open source for the community to download and verify.

The Core Technology: From 54 GB Down to Under 4 GB

PrismML's core value proposition lies in its proprietary model compression technique. Among conventional compression approaches, quantization reduces model weights from high-precision floating-point formats (e.g., FP32, BF16) to lower-precision integers (e.g., INT8, INT4), cutting storage requirements and speeding up inference at the cost of numerical precision. Pruning identifies and removes neurons or weight connections that contribute minimally to the output, but aggressive pruning similarly degrades performance. The two techniques are commonly used together, yet the industry broadly faces a compression-ratio vs. performance trade-off — shrinking a model almost always means sacrificing accuracy. PrismML claims its "mathematical tricks" can dramatically reduce model size while preserving performance to the greatest extent possible.

The numbers are striking: Qwen 3.6's original size is approximately 54 GB; after compression, it comes in at under 4 GB — a compression ratio of more than 13×. The technique stems from mathematical research at Caltech; PrismML's CEO, Babak Hassibi, is himself a professor of electrical engineering at the university, and the underlying patents are held by Caltech and exclusively licensed to PrismML.

Full Activation vs. Sparse Architecture: What's the Key Difference?

Understanding PrismML's claimed advantage requires attention to one critical technical detail — how parameters are activated.

Take Apple as an example. In its latest on-device AI solution, some iPhones run a local model with 20 billion parameters, but it uses a sparse architecture. The dominant technique behind this design is the Mixture of Experts (MoE) approach — the model is divided into multiple "expert" sub-networks, and a routing mechanism dynamically selects only a small number of experts for each inference pass. As a result, only 1 to 4 billion parameters are active at any given moment. MoE dramatically reduces per-inference compute, but at a cost: the knowledge encoded in the unselected experts is unavailable during that inference pass, which may limit contextual completeness for certain tasks.

By contrast, PrismML's model claims to have all 27 billion parameters activated simultaneously. Full parameter activation means every inference call draws on the entire knowledge network, theoretically offering stronger task generalization and more consistent reasoning — but it also imposes significantly higher demands on memory bandwidth and compute. Achieving this within the constrained hardware of a smartphone is where the claimed technical value lies — and it is precisely what external parties most need to independently verify.

"95% of Intelligence Will Run Locally"

In interviews, Hassibi painted a vision of a future dominated by on-device AI:

"Imagine a world, maybe three years from now, where 95% of the intelligence you need is available locally — on your phone, your laptop, your appliances — and only the last roughly 5% of high-end needs ever has to access the cloud. I think that's the direction people see things heading."

He further argued that shrinking models down to run on-device "fundamentally changes the economics of AI." This observation cuts to the heart of a key industry pain point: today's mainstream AI services rely on cloud inference, with leading AI companies spending tens of billions of dollars annually on inference costs. Network transmission introduces latency, with response times for complex tasks reaching several seconds. And all conversational data must leave the user's device, creating privacy and compliance risks. On-device inference moves computation to the user's hardware, fundamentally eliminating all three categories of cost — no cloud compute fees, no network latency, and no data leaving the device. If high-quality models can run locally, the marginal cost of AI services approaches zero, which has real appeal for both enterprises and individual users.

Ambitious Claims, but Key Questions Remain Unverified

PrismML's ambitions don't stop at 27 billion parameters. Hassibi says the company plans to keep pushing toward larger models, with trillion-parameter scale as the ultimate target. Trillion-parameter models currently represent the top tier of AI — notable examples include Google's Gemini Ultra and rumored versions of GPT-4, whose parameter counts are estimated to range from 500 billion to several trillion. Even the most advanced datacenter inference clusters today require tens to hundreds of high-end GPUs working in concert to run trillion-parameter models. The scale gap from 27 billion to one trillion parameters is more than 37×, and the compression and hardware adaptation challenges scale non-linearly. That said, Apple's Unified Memory Architecture in M-series chips, and the continuously improving NPU capabilities of Qualcomm Snapdragon and MediaTek Dimensity processors, are gradually turning "running top-tier models on-device" from science fiction into an engineering problem — which is the realistic foundation for what PrismML is pursuing.

Yet beyond the optimistic outlook, several key questions warrant careful scrutiny:

• Actual performance degradation: "Compression without performance loss" is an extraordinarily strong technical claim. The real-world benchmark performance of the compressed Qwen 3.6 will need independent community evaluation to verify.
• Real-world user experience: There is a meaningful difference between "runs" and "runs smoothly." Inference speed, thermal behavior, and battery impact of a fully-activated model on a smartphone remain unknown.
• Technical reproducibility: The details of the mathematical compression scheme have not been fully disclosed, and its generalizability to different model architectures has yet to be examined.

A Watershed Moment for On-Device AI?

Regardless of how evaluations ultimately play out, PrismML's effort reflects an increasingly clear industry trajectory: AI inference is migrating from the cloud to the edge. Apple, Qualcomm, Google, and other tech giants are already investing heavily in on-device AI, and PrismML represents a startup path into this space through extreme compression technology.

Once the open-source model is released, real-world community testing will serve as the true litmus test for its technical credibility. If PrismML's core claims survive independent verification, then "a 27-billion-parameter model running on an iPhone" may well mark a genuine milestone in the transition of mobile AI from a "feature demonstration" to a real productivity tool.

Key Takeaways