#AI inference acceleration
539 related articles
·3 min
From OpenCV to Industrial-Grade Vision: An Advanced CV Learning Path and Practical Guide
A complete advanced path from mastering OpenCV and YOLO basics to building industrial-grade computer vision systems, covering deep learning, custom model training, real-time inference, edge deployment, and spatial perception.
Read more →
·2 min
Google's SL2T Model: Real-Time Sign Language-to-Text Technology Arrives on Pixel 11
Google launches SL2T sign language-to-text model supporting real-time ASL-to-English conversion, integrated with Gboard and Live Transcribe, deploying on-device on Pixel 11 for system-level accessibility.
Read more →
·2 min
NVIDIA Open-Sources Switchyard: An AI Task Scheduling Engine Built with Rust
NVIDIA-NeMo team open-sources Switchyard, a high-performance AI task scheduling engine built in Rust. Explore its technical positioning, why Rust was chosen, and its strategic role in the NeMo ecosystem.
Read more →
·5 min
Harness: The Operating System That Takes LLM Agents from Demo to Production
Deep dive into Harness technology: how context engineering, memory management, and multi-agent architecture transform LLM agents from stochastic demos into stable production systems.
Read more →
·3 min
SAP Freezes Hiring and Travel: AI Costs Are Devouring Corporate Budgets
SAP freezes hiring and travel as AI spending surges, revealing the massive cost pressures enterprises face in AI transformation and how budgets are being reshaped.
Read more →
·2 min
NVIDIA Open-Sources Real-Time AI Animation Technology: Strategic Analysis and Developer Practice Guide
NVIDIA open-sources real-time AI animation tech for virtual streamers, game NPCs, and digital humans. Analysis of strategy, applications, and developer challenges.
Read more →
·3 min
Practical Guide to Accelerating LLM Inference with Apple Silicon Virtual Machines
A deep dive into accelerating llama.cpp inference in macOS VMs using Apple Silicon's unified memory architecture, covering Metal backend configuration, memory allocation, and quantization optimization.
Read more →
·2 min
Ododok: A Health Monitoring App That Tracks Chewing Count Using AirPods
Ododok uses AirPods' built-in sensors to track chewing count, rhythm, and meal duration in real time—no extra hardware needed to quantify eating behavior and build healthier habits.
Read more →
·3 min
PagedAttention Explained: GPU Memory Optimization and Model Routing Techniques
Deep dive into how PagedAttention borrows OS paging to optimize GPU memory, eliminate KV Cache fragmentation, and how model routing enables intelligent scheduling for multi-model inference services.
Read more →
·4 min
Needle2: How a 14MB On-Device Agentic LLM Could Transform Edge AI
Needle2 is a 14MB on-device agentic LLM designed for phones, wearables, smart homes, and robots. This article analyzes its compression techniques, architecture, and the cloud-to-edge AI paradigm shift.
Read more →
·3 min
Running Local LLMs on 24GB VRAM? Your Actual Usable Memory Is Less Than You Think
A detailed breakdown of actual usable VRAM when running local LLMs on 24GB GPUs. Covers the three memory buckets — model weights, KV cache, and runtime headroom — with structured planning methods.
Read more →
·3 min
Generating Minute-Long Videos Locally with MiniMax H3: A ComfyUI Context Loop Workflow Guide
Learn how to generate 1+ minute coherent long videos locally using MiniMax H3 with ComfyUI context loop nodes, covering frame passing, reference image consistency, and resolution-tiered debugging.
Read more →
·2 min
Ante: A Deep Dive into the Single-Binary Offline Programming AI Assistant
Ante is a fully offline single-binary programming AI agent requiring no internet or API keys. Deep analysis of its zero-dependency deployment, privacy protection, and cost advantages.
Read more →
·3 min
Stoa Markets: How a GPU and AI Server Marketplace Tackles the Compute Supply-Demand Challenge
Deep dive into how YC-backed Stoa Markets builds a GPU and AI server marketplace to solve compute fragmentation, price opacity, and supply-demand challenges.
Read more →
·2 min
SpeakoFlow: Open-Source Local Voice Assistant — A Privacy-First System-Wide Voice Input Tool for Desktop
SpeakoFlow is an open-source local voice assistant with system-wide voice input, screen understanding, and real-time translation. MIT-licensed, speech-to-text runs entirely locally to protect privacy. Supports Windows, macOS, and Linux.
Read more →
·2 min
Airy Voice Content Creation Tool: A Free, Fast, and Simple AI Voiceover Solution
Airy is a free, fast, and simple AI voice content creation tool. This article analyzes Airy's positioning, technology trends, market opportunities, and challenges in the lightweight voice creation space.
Read more →
·3 min
Real-Time Underwater Image Processing System: The Technology Behind 4K 60FPS
Deep dive into a real-time underwater image processing system running on a laptop, achieving 4K 60FPS color restoration via CUDA acceleration and an adaptive Sea-Thru engine, with HUD telemetry integrated on a FIFISH V-EVO ROV.
Read more →
·3 min
Gemini Omni and Nano Banana Model Analysis: Google's Multimodal AI Strategy
Analysis of Google's Gemini Omni full-modal model and Nano Banana lightweight model, exploring their positioning, technical features, and Google's multimodal AI product strategy.
Read more →
·4 min
Total Parameter Count Is Obsolete: Understanding the Two-Number Era of MoE Models
Mixture-of-Experts (MoE) makes single parameter counts obsolete. Learn the difference between total and active parameters, how MoE decouples knowledge capacity from inference cost, and why this matters for model selection.
Read more →
·4 min
The Shape and Trajectory of Technological Evolution: How to Understand and Predict Future Tech Trends
Deep analysis of the underlying logic and key trends in technological evolution, covering AI infrastructure, computing paradigm shifts, and human-machine collaboration, with frameworks for developers and entrepreneurs.
Read more →