#on-device LLM
118 related articles
·3 min
Decoding Apple's AI Strategy: A Masterstroke of Patience or a Strategic Misstep?
A deep analysis of Apple's restrained AI strategy: historical fast-follower patterns, bubble-bursting logic, hardware moat advantages, and the risks of waiting too long.
Read more →
·5 min
Why Are China's Open-Source AI Models Getting Stronger? A Deep Dive Into Three Comeback Strategies
Facing US chip bans and closed-source monopoly, how do China's open-source AI models keep fighting back? A deep dive into three core paths: open-source pricing games, optical interconnect positioning, and on-device scenarios.
Read more →
·3 min
Zhipu's Mobile AI Agent Tested: A Full Breakdown of the Cloud Phone Approach — Capabilities and Limitations
Hands-on test of Zhipu's mobile AI Agent: using a cloud phone to bypass permission limits, it supports natural language-driven automation. We cover its core mechanics, real performance, app restrictions, and future potential.
Read more →
ChatGPT Work Deep Dive: The Cloud-Loca…
·3 min
ChatGPT Work Deep Dive: The Cloud-Local Split and What It Means for Users
ChatGPT Work runs in the cloud on web/mobile but accesses local files on desktop — and they don't sync. A deep dive into the split design, UX tradeoffs, and broader AI agent challenges.
Read more →
·3 min
Qwen3 Deep Dive: A Complete Look at the Open-Source Hybrid Reasoning Model
Qwen3 deep dive: 6 Dense & MoE models from 0.6B to 235B, world's first open-source hybrid reasoning model, matching Gemini 2.5 Pro. Complete guide for developers and enterprises.
Read more →
Moonshine: A Low-Latency Speech Engine…
·2 min
Moonshine: A Low-Latency Speech Engine and Open-Source Powerhouse for Building Voice Agents
Moonshine is an open-source, C++-based low-latency speech engine combining STT, intent recognition, and TTS for building voice agents. 9,400+ GitHub Stars.
Read more →
·3 min
Claude Opus 5 Launch Imminent, Alibaba Qwen Officially Integrated into Apple Intelligence
Claude Opus 5 launches next week; Alibaba Qwen integrates into Apple Intelligence for Chinese users; 27B on-device model compressed to 3.8GB; open-source models narrow gap to closed-source by 3.3%.
Read more →
·3 min
LightMem-Ego: The AI Memory Assistant That Remembers the Details of Your Daily Life
LightMem-Ego is an AI life assistant with long-term memory — tracking object locations, meetings, and life events over time. A deep dive into its technology and real-world challenges.
Read more →
·3 min
DeepSeek and Zhipu Race to Build Custom AI Chips: China's Path to AI Compute Independence
DeepSeek seeks $7B for custom AI inference chips; Zhipu AI explores ASIC. Deep dive into China's AI compute independence strategy, multimodal generation, agents, and hardware trends.
Read more →
AirLLM: How a 4GB GPU Can Run a 70B Mo…
·2 min
AirLLM: How a 4GB GPU Can Run a 70B Model — Core Principles and Use Cases
AirLLM is an open-source project that uses layer-by-layer inference to run 70B LLMs on a single 4GB GPU. Learn how it works, its tradeoffs, and ideal use cases.
Read more →
·3 min
GPT-5.6 Launches, Siri Gets a Major Overhaul: Your Complete AI Industry Roundup
OpenAI launches GPT-5.6, Apple rebuilds Siri, China's AI companionship regulations take effect, Google labels AI images — your one-stop global AI industry roundup.
Read more →
Block Low-Rank Compression: A Guide to…
·2 min
Block Low-Rank Compression: A Guide to Accelerating Large Model Inference on Memory-Constrained GPUs
Learn how Block Low-Rank (BLR) decomposition compresses large model memory usage and accelerates GPU inference, including CUDA kernel optimization and combination with quantization and pruning.
Read more →
The Guardian Angels Framework: How LLM…
·3 min
The Guardian Angels Framework: How LLM Personalization Can Achieve Both Efficiency and Data Security
The Guardian Angels framework shows how LLM personalization can achieve both productivity and data security through local deployment, differential privacy, and tiered permissions.
Read more →
·3 min
Chrome Built-in AI in Action: Six APIs for Building a Smart Blog Editor
Deep dive into Chrome Built-in AI's six core APIs — Summarizer, Prompt API, Translator and more — applied to a real blog editor. Local, private, no server needed.
Read more →
·3 min
The AI Privacy Debate: Why Local Models and Open-Source Frameworks Are Essential
When cloud AI privacy terms hide data-sharing risks, local model deployment and open-source frameworks offer developers a path to true data control. Analyzing xAI, OpenAI, GLM, Kimi, and Qwen.
Read more →
·2 min
Local AI Coding in Android Studio: Running Gemma Models Offline
Google demoed running Gemma local LLMs directly in Android Studio at I/O Connect, enabling offline AI coding with full privacy — no code leaves your machine.
Read more →
BillAI Bass: A Complete Guide to Rebui…
·2 min
BillAI Bass: A Complete Guide to Rebuilding a Classic Novelty Toy with an AI Agent
A developer rebuilt Big Mouth Billy Bass into an AI assistant using AWS Strands Agents, enabling real-time voice conversation with synced physical movements.
Read more →
·4 min
AI Text Detection and Privacy Protection: The Dual Dilemma of LLM Security
How does watermarking work — and why won't companies deploy it? How does differential privacy defend against membership inference attacks? Based on talks by IISc and IIT scholars, this article unpacks the core mechanisms and real challenges in LLM security.
Read more →
·3 min
31 Companies Sign Agent Privacy Pact: Industry Self-Regulation Framework Goes into Effect
31 companies sign China's first AI agent privacy pact covering screen-reading authorization, training data restrictions, and payment caps. Plus: domestic LLM tops OpenRouter, Meta's $40B compute bet, and agent commercialization challenges.
Read more →
PrismML Breakthrough: How a 27-Billion…
·2 min
PrismML Breakthrough: How a 27-Billion-Parameter LLM Fits Inside an iPhone
PrismML compressed Alibaba's Qwen 3.6 from 54 GB to under 4 GB, enabling a fully-activated 27B-parameter model to run locally on iPhone 17 Pro. Here's how.
Read more →