#ONNX Runtime
44 related articles
·3 min
Running a Local AI Voice Assistant on a $50 Arduino Board: A Practical Guide to Edge AI
How to run a fully local AI voice agent on a $50 Arduino Uno Q board, covering speech recognition, intent understanding, and TTS implementation for edge AI applications.
Read more →
·3 min
Hand Wave: Real-Time Sign Language Translation on Meta Smart Glasses for Barrier-Free Communication
Hand Wave uses Meta smart glasses' camera and open-source AI neural networks to translate sign language into text and speech in real time, supporting iOS and Web platforms.
Read more →
·3 min
CutWire Drift: An Open-Source Video Editing Tool Powered by Local AI
CutWire Drift is a beginner-friendly open-source video editor with local AI features including Whisper auto-subtitles, SAM2 background removal, multi-track timeline, keyframe animation, and transitions—free and privacy-preserving.
Read more →
·2 min
Building Custom C/C++ Inference Engines: A Deep Dive into the Tradeoffs Between Performance, Control, and Engineering Cost
Deep analysis of why teams build custom C/C++ inference engines instead of using PyTorch or TensorRT, exploring performance, minimal dependencies, and long-term maintenance tradeoffs.
Read more →
·3 min
RearAware: A Local AI That Blurs Cat Butts Reveals the Dataset Dilemma
RearAware is a local AI Chrome extension that detects and blurs cat butts in video calls. This article analyzes its niche dataset challenges and explores solutions like augmentation, synthetic data, and transfer learning.
Read more →
·2 min
Is Being a Computer Vision Engineer Worth It? A Deep Dive into Salary and Career Paths
In-depth analysis comparing CV engineer vs. standard SDE salaries, career growth, and satisfaction. Explore the advantages and market limitations of specializing in computer vision.
Read more →
·3 min
PyTorch vs TensorFlow: How Beginners Should Choose a Deep Learning Framework
Should deep learning beginners choose PyTorch or TensorFlow? This article compares both frameworks on research trends, ecosystem, and deployment, with practical switching advice.
Read more →
·3 min
Fix for Immich Crashes Caused by Proxmox's Default kvm64 CPU Type
Proxmox's default kvm64 CPU type only exposes x86-64-v1 instructions, causing Immich's ML container to crash when NumPy's baseline isn't met. Learn the root cause and one-command fix.
Read more →
·3 min
ncnn Vulkan Cross-Platform Edge Inference in Practice: Solving the GPU Fragmentation Deployment Challenge
Facing GPU fragmentation on edge devices, the PostSlate team used ncnn's Vulkan backend for cross-platform ML inference, achieving 10× speedup on RTX 4070 with half the model size and zero runtime installation.
Read more →
·3 min
Monocular RGB Human Mesh Reconstruction: Real-Time Motion Capture at 18 FPS with Rust + CUDA
A deep dive into a real-time 3D human mesh reconstruction project using a single RGB camera, built with Rust, Candle, and CUDA, achieving 55ms/frame on an RTX 5080.
Read more →
·3 min
Monocular RGB Human Mesh Reconstruction: Real-Time Motion Capture at 18 FPS with Rust + CUDA
Deep dive into a real-time 3D human mesh reconstruction project using a single RGB camera, built with Rust, Candle, and CUDA, achieving 55ms/frame on RTX 5080. Exploring its architecture, Metal porting plans, and applications in VTuber, AR/VR, and sports analysis.
Read more →
·3 min
Self-Hosting Hardware Upgrade Guide: From an Old NUC to a Game Server and Beyond
A detailed guide to self-hosting hardware upgrades: analyzing NUC performance bottlenecks, comparing used enterprise Mini PCs and custom ITX builds for Jellyfin, Immich, and Minecraft servers.
Read more →
·3 min
Self-Hosting Hardware Upgrade Guide: From an Old NUC to a Game Server and Beyond
A detailed guide to self-hosting hardware upgrades: analyzing NUC bottlenecks, comparing used enterprise Mini PCs and ITX builds for Jellyfin, Immich, and Minecraft servers.
Read more →
·3 min
Building a YOLO26 Inference Engine from Scratch in ARM64 Assembly: A Deep Dive into Low-Level Optimization
Deep dive into building a YOLO26n object detection inference engine from scratch using ARM64 assembly and C, covering NEON SIMD, Winograd convolution, GEMM micro-kernels, and cache tiling optimizations.
Read more →
Intelligent Model Routing: The Core Te…
·3 min
Intelligent Model Routing: The Core Technology and Implementation Guide for AI Cost Reduction and Efficiency
Intelligent Model Routing is becoming key AI infrastructure. This article explores its principles, solution types, technical challenges, and implementation considerations to help developers balance cost, latency, and quality.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment
A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics on Model Quantization and Inference Deployment
A systematic guide to must-know AI application engineer interview topics: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
Moonshine: A Low-Latency Speech Engine…
·2 min
Moonshine: A Low-Latency Speech Engine and Open-Source Powerhouse for Building Voice Agents
Moonshine is an open-source, C++-based low-latency speech engine combining STT, intent recognition, and TTS for building voice agents. 9,400+ GitHub Stars.
Read more →
·3 min
Bernini Tile-Based Generation: An Engineered Approach to Video Super-Resolution in ComfyUI
Bernini is a ComfyUI video super-resolution node package using Tile Split/Select/Merge to solve seams, drift, and VRAM overflow. Benchmarked at 325s for 39 frames at 1920×1080.
Read more →
·2 min
SentinelCV: An Open-Source AI Fall Detection Solution Using Your Existing Cameras
SentinelCV is an open-source YOLOv8-based fall detection system that works with existing CCTV and IP cameras. Get real-time alerts via Telegram — no new hardware needed.
Read more →