#on-device inference
140 related articles
·3 min
NVIDIA ACE SDK: On-Device AI Inference for Intelligent Game NPC Companions
Deep dive into NVIDIA ACE Game Agent SDK's integration with Unreal Engine 5, exploring how on-device AI inference enables low-latency, privacy-safe intelligent NPC dialogue and behavior.
Read more →
·3 min
Firebase's Major Upgrade: Real-Time SQL Sync, On-Device AI Inference, and Agent Skills
Firebase unveils major updates at Google I/O 2025: SQL Connect real-time sync, AI Logic on-device inference, Firestore Enterprise full-text search, phone verification GA, and more.
Read more →
·4 min
Google Hybrid Inference Comes to iOS: A Complete Guide to On-Device AI Cross-Platform Deployment
Google Hybrid Inference officially supports iOS, adds Gemma 4 on Android, and Chrome local Web inference nears GA. A deep dive into hybrid inference technology, cross-platform advantages, and developer opportunities.
Read more →
·4 min
Google Hybrid Inference Comes to iOS: A Complete Guide to On-Device AI Cross-Platform Deployment
Google Hybrid Inference now supports iOS, adds Gemma 4 on Android, and Chrome local Web inference nears GA. A deep dive into hybrid inference technology, cross-platform advantages, and developer opportunities.
Read more →
·2 min
Chrome Built-in AI Explained: How Locally Running Models Protect Your Privacy
Explore Chrome Built-in AI technology and how running AI models locally in the browser enables zero data upload, instant responses, and stronger privacy protection.
Read more →
·2 min
Chrome Built-in AI Explained: How Locally Running Models Protect Your Privacy
Deep dive into Chrome Built-in AI technology, exploring how running AI models locally in the browser achieves zero data uploads, instant responses, and stronger privacy protection.
Read more →
·3 min
Apple Silicon LLM Inference Speed Benchmarks: A Complete Guide to Local Large Model Performance on Mac
In-depth analysis of Apple Silicon local LLM inference speed benchmarks covering M-series memory bandwidth, model quantization, MLX framework optimization, and Mac configuration guidance.
Read more →
·3 min
The Computer Is Being Redefined in the AI Era — Has Your Understanding Kept Up?
Generative AI is profoundly redefining the personal computer — from passive tool to intelligent collaborator. This article examines the core shifts of the AI PC era and the productivity gap created by cognitive lag.
Read more →
·3 min
Computers Are Being Redefined in the AI Era — Has Your Understanding Kept Up?
Generative AI is profoundly redefining personal computers — from passive tools to intelligent collaborators, from deterministic computation to probabilistic reasoning. Explore the core shifts of the AI PC era.
Read more →
·3 min
Decoding Apple's AI Strategy: A Masterstroke of Patience or a Strategic Misstep?
A deep analysis of Apple's restrained AI strategy: historical fast-follower patterns, bubble-bursting logic, hardware moat advantages, and the risks of waiting too long.
Read more →
·1 min
Radxa Cubie A7Z Real-World Test: 330ms End-to-End Object Detection Latency Under Full NPU Load
Deep analysis of Radxa Cubie A7Z achieving 330ms glass-to-glass object detection latency under full NPU load, covering latency breakdown, practical use cases, and edge AI optimization strategies.
Read more →
·2 min
Google Flash and Flash-Lite Models Upgraded and Live: A New Lightweight, High-Efficiency AI Option
Google rolls out upgraded 3.6 Flash and 3.5 Flash-Lite models. Learn about the Flash series' positioning, upgrade highlights, and value for developers.
Read more →
·5 min
Why Are China's Open-Source AI Models Getting Stronger? A Deep Dive Into Three Comeback Strategies
Facing US chip bans and closed-source monopoly, how do China's open-source AI models keep fighting back? A deep dive into three core paths: open-source pricing games, optical interconnect positioning, and on-device scenarios.
Read more →
ChatGPT Work Deep Dive: The Cloud-Loca…
·3 min
ChatGPT Work Deep Dive: The Cloud-Local Split and What It Means for Users
ChatGPT Work runs in the cloud on web/mobile but accesses local files on desktop — and they don't sync. A deep dive into the split design, UX tradeoffs, and broader AI agent challenges.
Read more →
·2 min
OpenAI Acquires Jony Ive's Company io for $6.5 Billion, Betting Big on the AI Hardware Era
OpenAI acquires Jony Ive's hardware startup io for ~$6.5B in its largest deal ever, signaling a bold push into AI-native consumer hardware beyond the smartphone era.
Read more →
·3 min
27B Parameter LLM on an iPhone? A Deep Dive into On-Device Full-Parameter AI
Startup Prismo claims to compress a 27B-parameter model to 4GB for full local inference on iPhone 17 Pro. We break down the compression tech, compare it to Apple's MoE approach, and assess real-world limits.
Read more →
·3 min
Qwen3 Deep Dive: A Complete Look at the Open-Source Hybrid Reasoning Model
Qwen3 deep dive: 6 Dense & MoE models from 0.6B to 235B, world's first open-source hybrid reasoning model, matching Gemini 2.5 Pro. Complete guide for developers and enterprises.
Read more →
Moonshine: A Low-Latency Speech Engine…
·2 min
Moonshine: A Low-Latency Speech Engine and Open-Source Powerhouse for Building Voice Agents
Moonshine is an open-source, C++-based low-latency speech engine combining STT, intent recognition, and TTS for building voice agents. 9,400+ GitHub Stars.
Read more →
·3 min
Claude Opus 5 Launch Imminent, Alibaba Qwen Officially Integrated into Apple Intelligence
Claude Opus 5 launches next week; Alibaba Qwen integrates into Apple Intelligence for Chinese users; 27B on-device model compressed to 3.8GB; open-source models narrow gap to closed-source by 3.3%.
Read more →
Local AI Models vs. Cloud: This Tech J…
·2 min
Local AI Models vs. Cloud: This Tech Judgment Call Is Testing Your Thinking
"Your opinion on local AI is an IQ test" — this viral claim reveals the cognitive divide between local and cloud AI deployment, from data sovereignty to TCO.
Read more →