#ONNX
61 related articles
·6 min
14x Speedup: How Manticore Rebuilt the ONNX Embedding Inference Path
Manticore Search restructured its ONNX inference path to achieve 14x faster text embeddings. Deep dive into batching, session reuse, zero-copy memory, and thread tuning for vector search systems.
Read more →
Tutorials·2 min
ONNX Model Deployment in Practice: A Complete Guide from PyTorch Export to Cross-Framework Inference
Complete guide to ONNX neural network exchange format: PyTorch and TensorFlow model export, ONNX Runtime inference, MNIST classifier deployment, and HuggingFace model downloads with runnable code examples.
Read more →
·3 min
Hand Wave: Real-Time Sign Language Translation on Meta Smart Glasses for Barrier-Free Communication
Hand Wave uses Meta smart glasses' camera and open-source AI neural networks to translate sign language into text and speech in real time, supporting iOS and Web platforms.
Read more →
·2 min
Inventory: Locally Search All Your AI Conversation History
Inventory is a local-first AI conversation search tool that unifies search across Cursor, Claude Code, Zed, Codex, and Kiro. No signups, no cloud, one-time purchase.
Read more →
·3 min
CutWire Drift: An Open-Source Video Editing Tool Powered by Local AI
CutWire Drift is a beginner-friendly open-source video editor with local AI features including Whisper auto-subtitles, SAM2 background removal, multi-track timeline, keyframe animation, and transitions—free and privacy-preserving.
Read more →
·2 min
Building Custom C/C++ Inference Engines: A Deep Dive into the Tradeoffs Between Performance, Control, and Engineering Cost
Deep analysis of why teams build custom C/C++ inference engines instead of using PyTorch or TensorRT, exploring performance, minimal dependencies, and long-term maintenance tradeoffs.
Read more →
·3 min
RearAware: A Local AI That Blurs Cat Butts Reveals the Dataset Dilemma
RearAware is a local AI Chrome extension that detects and blurs cat butts in video calls. This article analyzes its niche dataset challenges and explores solutions like augmentation, synthetic data, and transfer learning.
Read more →
·2 min
Is Being a Computer Vision Engineer Worth It? A Deep Dive into Salary and Career Paths
In-depth analysis comparing CV engineer vs. standard SDE salaries, career growth, and satisfaction. Explore the advantages and market limitations of specializing in computer vision.
Read more →
·3 min
PyTorch vs TensorFlow: How Beginners Should Choose a Deep Learning Framework
Should deep learning beginners choose PyTorch or TensorFlow? This article compares both frameworks on research trends, ecosystem, and deployment, with practical switching advice.
Read more →
·3 min
Fix for Immich Crashes Caused by Proxmox's Default kvm64 CPU Type
Proxmox's default kvm64 CPU type only exposes x86-64-v1 instructions, causing Immich's ML container to crash when NumPy's baseline isn't met. Learn the root cause and one-command fix.
Read more →
·3 min
ncnn Vulkan Cross-Platform Edge Inference in Practice: Solving the GPU Fragmentation Deployment Challenge
Facing GPU fragmentation on edge devices, the PostSlate team used ncnn's Vulkan backend for cross-platform ML inference, achieving 10× speedup on RTX 4070 with half the model size and zero runtime installation.
Read more →
·3 min
The Real Daily Life of MLE and MLOps Engineers: Key Differences Between Three Roles and Career Paths
An in-depth look at the real daily work of data scientists, MLEs, and MLOps engineers — covering responsibilities, essential tools, and career paths to help you find your direction in AI.
Read more →
·3 min
Monocular RGB Human Mesh Reconstruction: Real-Time Motion Capture at 18 FPS with Rust + CUDA
Deep dive into a real-time 3D human mesh reconstruction project using a single RGB camera, built with Rust, Candle, and CUDA, achieving 55ms/frame on RTX 5080. Exploring its architecture, Metal porting plans, and applications in VTuber, AR/VR, and sports analysis.
Read more →
·3 min
Monocular RGB Human Mesh Reconstruction: Real-Time Motion Capture at 18 FPS with Rust + CUDA
A deep dive into a real-time 3D human mesh reconstruction project using a single RGB camera, built with Rust, Candle, and CUDA, achieving 55ms/frame on an RTX 5080.
Read more →
·3 min
Self-Hosting Hardware Upgrade Guide: From an Old NUC to a Game Server and Beyond
A detailed guide to self-hosting hardware upgrades: analyzing NUC bottlenecks, comparing used enterprise Mini PCs and ITX builds for Jellyfin, Immich, and Minecraft servers.
Read more →
·3 min
Self-Hosting Hardware Upgrade Guide: From an Old NUC to a Game Server and Beyond
A detailed guide to self-hosting hardware upgrades: analyzing NUC performance bottlenecks, comparing used enterprise Mini PCs and custom ITX builds for Jellyfin, Immich, and Minecraft servers.
Read more →
·3 min
Building a YOLO26 Inference Engine from Scratch in ARM64 Assembly: A Deep Dive into Low-Level Optimization
Deep dive into building a YOLO26n object detection inference engine from scratch using ARM64 assembly and C, covering NEON SIMD, Winograd convolution, GEMM micro-kernels, and cache tiling optimizations.
Read more →
Intelligent Model Routing: The Core Te…
·3 min
Intelligent Model Routing: The Core Technology and Implementation Guide for AI Cost Reduction and Efficiency
Intelligent Model Routing is becoming key AI infrastructure. This article explores its principles, solution types, technical challenges, and implementation considerations to help developers balance cost, latency, and quality.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment
A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics on Model Quantization and Inference Deployment
A systematic guide to must-know AI application engineer interview topics: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →