#on-device LLM
109 related articles
·3 min
NVIDIA ACE SDK: On-Device AI Inference for Intelligent Game NPC Companions
Deep dive into NVIDIA ACE Game Agent SDK's integration with Unreal Engine 5, exploring how on-device AI inference enables low-latency, privacy-safe intelligent NPC dialogue and behavior.
Read more →
·3 min
Making LLMs Faster and Lighter: A Practical Approach to Reshaping Sparsity for GPUs
Deep dive into Sakana AI and NVIDIA's latest research using TwELL sparse packing format and custom CUDA kernels to convert LLM sparsity into real GPU speedups, achieving 20%+ faster inference/training and significantly lower memory usage.
Read more →
·3 min
PilotDeck: A Local Console That Tames Multi-Task Agent Chaos
PilotDeck is an open-source local Agent console from a Tsinghua-affiliated team that solves multi-task chaos with workspace isolation, white-box memory management, and smart model routing.
Read more →
·3 min
Firebase AI Logic Deep Dive: Hybrid Inference, Prompt Templates & Multi-Layer Security Deployment
Deep dive into Firebase AI Logic: server-side prompt templates to prevent leakage, Cloud Function triggers, four-layer security defense, AI monitoring with context caching for cost control, and cross-platform hybrid inference.
Read more →
·3 min
Firebase's Major Upgrade: Real-Time SQL Sync, On-Device AI Inference, and Agent Skills
Firebase unveils major updates at Google I/O 2025: SQL Connect real-time sync, AI Logic on-device inference, Firestore Enterprise full-text search, phone verification GA, and more.
Read more →
·3 min
HarmonyOS 7 Developer Beta Launches: A Deep Dive into System-Level Transformation for the Agent Era
HarmonyOS 7 developer Beta launches, claiming to be the world's first AI-native OS. Deep analysis of Xiaoyi Agent, StarShield security, Galaxy Interconnect, and the OS AI competition landscape.
Read more →
·1 min
Gemma 4 12B: Google's Open-Weight Model Runs Locally on Your Laptop
Google releases Gemma 4 12B, an open-weight model that runs locally on laptops. Learn about its performance, local deployment value, and the open-source LLM competitive landscape.
Read more →
·2 min
Jeff Dean's Commencement Speech at UW Allen School: A Message to the Next Generation of Engineers in the AI Era
Jeff Dean delivers commencement speech at UW Allen School of Computer Science & Engineering, sharing insights with the next generation of CS graduates in the AI era.
Read more →
·3 min
Gemma 4 Open-Source Model Local Deployment Guide: Ollama Installation & Mobile Setup
Step-by-step guide to deploying Google's Gemma 4 open-source model locally with Ollama and running the lightweight version on mobile with tool calling support.
Read more →
·2 min
iOS 27 Should Give Siri AI Home Screen Widgets — Stop Falling Further Behind ChatGPT and Others
ChatGPT, Gemini, and Claude all have iOS home screen widgets, but Siri still relies on voice activation. Here's why Siri needs a home screen shortcut and what an ideal iOS 27 Siri widget should do.
Read more →
·2 min
New Siri Mac Experience: Earning Rave Reviews Even at the Beta 1 Stage
A tech user bypassed the waitlist via Terminal to try the new Siri on Mac. Even at Beta 1, the verdict was "Apple nailed it" — a promising sign for Apple Intelligence.
Read more →
·2 min
iOS App Publishing in Practice: From Device Testing to Certificates, Signing, and App Filing — The Complete Workflow
Complete guide to publishing an iOS app: on-device debugging, Apple Distribution certificates, China-region App filing, privacy policies, and more for indie developers.
Read more →
·2 min
Apple Rarely Labels New Siri as Beta — What's Behind the Decision?
Apple has internally labeled the new Siri as Beta — an extremely rare move. This article analyzes the technical reasoning, waitlist mechanism, and the industry shift from perfectionism to iterationism.
Read more →
·2 min
Three Key Narrative Threads Behind Siri's Reboot
Apple's Siri is undergoing its biggest reboot ever. Tim Cook sets AI strategy, Craig Federighi leads integration, and Mike Rockwell rebuilds Siri from the ground up.
Read more →
·2 min
CreateNow Agent Configuration Tutorial: One-Click Integration with DeepSeek and Other Major LLMs
Step-by-step guide to configuring CreateNow agents with DeepSeek, Kimi, and Xiaomi LLMs. Covers one-click setup, custom model integration, and API Key acquisition for building AI digital employees.
Read more →
·3 min
AI Large Language Model Learning Roadmap for Beginners: A Three-Month Hands-On Guide
A detailed zero-to-hero AI large model learning roadmap covering four phases—fundamentals, RAG, Agents, and engineering deployment—with a practical three-month study plan and career advice.
Read more →
·3 min
Cursor Design Mode Launch and OpenAI Codex Updates: Latest Developments in AI Programming Tools
Cursor launches Design Mode for visual development, OpenAI Codex updates and Safety Lock Mode released, Anthropic doubles limits, AI agent leaderboards debut, Google DeepMind model compression breakthrough.
Read more →
·4 min
Google Hybrid Inference Comes to iOS: A Complete Guide to On-Device AI Cross-Platform Deployment
Google Hybrid Inference officially supports iOS, adds Gemma 4 on Android, and Chrome local Web inference nears GA. A deep dive into hybrid inference technology, cross-platform advantages, and developer opportunities.
Read more →
·4 min
Google Hybrid Inference Comes to iOS: A Complete Guide to On-Device AI Cross-Platform Deployment
Google Hybrid Inference now supports iOS, adds Gemma 4 on Android, and Chrome local Web inference nears GA. A deep dive into hybrid inference technology, cross-platform advantages, and developer opportunities.
Read more →
·4 min
Firebase AI Logic Update: Expanded Model Support and Enhanced Output Integrity Explained
Firebase AI Logic gets major updates at Google I/O, expanding AI model support and enhancing output integrity. Learn how these changes impact developers.
Read more →