#on-device LLM
83 related articles
·3 min
27B Parameter LLM on an iPhone? A Deep Dive into On-Device Full-Parameter AI
Startup Prismo claims to compress a 27B-parameter model to 4GB for full local inference on iPhone 17 Pro. We break down the compression tech, compare it to Apple's MoE approach, and assess real-world limits.
Read more →
·2 min
On-Device LLMs, Custom Silicon, and 100K-GPU Clusters: AI Industry Enters Full-Spectrum Competition
Apple's 1-bit compression fits 27B models on iPhone, Meta builds custom chip Iris, China's 100K-GPU cluster goes live, Samsung enters AI PC — a deep dive into AI's new full-spectrum competition.
Read more →
Bonsai 27B: The First 1-bit LLM That R…
·3 min
Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI
Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.
Read more →
·3 min
Apple Silicon LLM Inference Speed Benchmarks: A Complete Guide to Local Large Model Performance on Mac
In-depth analysis of Apple Silicon local LLM inference speed benchmarks covering M-series memory bandwidth, model quantization, MLX framework optimization, and Mac configuration guidance.
Read more →
·3 min
The Computer Is Being Redefined in the AI Era — Has Your Understanding Kept Up?
Generative AI is profoundly redefining the personal computer — from passive tool to intelligent collaborator. This article examines the core shifts of the AI PC era and the productivity gap created by cognitive lag.
Read more →
·3 min
Computers Are Being Redefined in the AI Era — Has Your Understanding Kept Up?
Generative AI is profoundly redefining personal computers — from passive tools to intelligent collaborators, from deterministic computation to probabilistic reasoning. Explore the core shifts of the AI PC era.
Read more →
·3 min
Decoding Apple's AI Strategy: A Masterstroke of Patience or a Strategic Misstep?
A deep analysis of Apple's restrained AI strategy: historical fast-follower patterns, bubble-bursting logic, hardware moat advantages, and the risks of waiting too long.
Read more →
·5 min
Why Are China's Open-Source AI Models Getting Stronger? A Deep Dive Into Three Comeback Strategies
Facing US chip bans and closed-source monopoly, how do China's open-source AI models keep fighting back? A deep dive into three core paths: open-source pricing games, optical interconnect positioning, and on-device scenarios.
Read more →
·3 min
Zhipu's Mobile AI Agent Tested: A Full Breakdown of the Cloud Phone Approach — Capabilities and Limitations
Hands-on test of Zhipu's mobile AI Agent: using a cloud phone to bypass permission limits, it supports natural language-driven automation. We cover its core mechanics, real performance, app restrictions, and future potential.
Read more →
ChatGPT Work Deep Dive: The Cloud-Loca…
·3 min
ChatGPT Work Deep Dive: The Cloud-Local Split and What It Means for Users
ChatGPT Work runs in the cloud on web/mobile but accesses local files on desktop — and they don't sync. A deep dive into the split design, UX tradeoffs, and broader AI agent challenges.
Read more →
·3 min
Qwen3 Deep Dive: A Complete Look at the Open-Source Hybrid Reasoning Model
Qwen3 deep dive: 6 Dense & MoE models from 0.6B to 235B, world's first open-source hybrid reasoning model, matching Gemini 2.5 Pro. Complete guide for developers and enterprises.
Read more →
Moonshine: A Low-Latency Speech Engine…
·2 min
Moonshine: A Low-Latency Speech Engine and Open-Source Powerhouse for Building Voice Agents
Moonshine is an open-source, C++-based low-latency speech engine combining STT, intent recognition, and TTS for building voice agents. 9,400+ GitHub Stars.
Read more →
·3 min
Claude Opus 5 Launch Imminent, Alibaba Qwen Officially Integrated into Apple Intelligence
Claude Opus 5 launches next week; Alibaba Qwen integrates into Apple Intelligence for Chinese users; 27B on-device model compressed to 3.8GB; open-source models narrow gap to closed-source by 3.3%.
Read more →
·3 min
LightMem-Ego: The AI Memory Assistant That Remembers the Details of Your Daily Life
LightMem-Ego is an AI life assistant with long-term memory — tracking object locations, meetings, and life events over time. A deep dive into its technology and real-world challenges.
Read more →
·3 min
DeepSeek and Zhipu Race to Build Custom AI Chips: China's Path to AI Compute Independence
DeepSeek seeks $7B for custom AI inference chips; Zhipu AI explores ASIC. Deep dive into China's AI compute independence strategy, multimodal generation, agents, and hardware trends.
Read more →
AirLLM: How a 4GB GPU Can Run a 70B Mo…
·2 min
AirLLM: How a 4GB GPU Can Run a 70B Model — Core Principles and Use Cases
AirLLM is an open-source project that uses layer-by-layer inference to run 70B LLMs on a single 4GB GPU. Learn how it works, its tradeoffs, and ideal use cases.
Read more →
·3 min
GPT-5.6 Launches, Siri Gets a Major Overhaul: Your Complete AI Industry Roundup
OpenAI launches GPT-5.6, Apple rebuilds Siri, China's AI companionship regulations take effect, Google labels AI images — your one-stop global AI industry roundup.
Read more →
Block Low-Rank Compression: A Guide to…
·2 min
Block Low-Rank Compression: A Guide to Accelerating Large Model Inference on Memory-Constrained GPUs
Learn how Block Low-Rank (BLR) decomposition compresses large model memory usage and accelerates GPU inference, including CUDA kernel optimization and combination with quantization and pruning.
Read more →
The Guardian Angels Framework: How LLM…
·3 min
The Guardian Angels Framework: How LLM Personalization Can Achieve Both Efficiency and Data Security
The Guardian Angels framework shows how LLM personalization can achieve both productivity and data security through local deployment, differential privacy, and tiered permissions.
Read more →
·3 min
Chrome Built-in AI in Action: Six APIs for Building a Smart Blog Editor
Deep dive into Chrome Built-in AI's six core APIs — Summarizer, Prompt API, Translator and more — applied to a real blog editor. Local, private, no server needed.
Read more →