#TensorRT inference
111 related articles
·3 min
Building a YOLO26 Inference Engine from Scratch in ARM64 Assembly: A Deep Dive into Low-Level Optimization
Deep dive into building a YOLO26n object detection inference engine from scratch using ARM64 assembly and C, covering NEON SIMD, Winograd convolution, GEMM micro-kernels, and cache tiling optimizations.
Read more →
·3 min
Opus 5 Hands-On: Powerful but Slow — Inference Speed Becomes the New AI Battleground
An AI practitioner tests Opus 5: powerful capabilities but painfully slow inference. Deep analysis of how LLM competition is shifting from intelligence to inference speed.
Read more →
·7 min
DIY Companion Robot: Building Your AI Desktop Buddy with Jetson Orin
A maker builds a DIY companion robot with NVIDIA Jetson Orin and 4S LiPo battery. Explore the full development journey from first power-up to AI interaction, including edge computing, power design, and companion robot trends.
Read more →
Intelligent Model Routing: The Core Te…
·3 min
Intelligent Model Routing: The Core Technology and Implementation Guide for AI Cost Reduction and Efficiency
Intelligent Model Routing is becoming key AI infrastructure. This article explores its principles, solution types, technical challenges, and implementation considerations to help developers balance cost, latency, and quality.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment
A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics on Model Quantization and Inference Deployment
A systematic guide to must-know AI application engineer interview topics: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·3 min
Jetson Edge AI Development: Setting Up a VS Code + Claude Code Workstation
Set up an AI coding workstation on NVIDIA Jetson using VS Code + Claude Code. Covers JetPack NVIDIA Skills, token budget management, headless SSH dev, and coding agent best practices.
Read more →
Moonshine: A Low-Latency Speech Engine…
·2 min
Moonshine: A Low-Latency Speech Engine and Open-Source Powerhouse for Building Voice Agents
Moonshine is an open-source, C++-based low-latency speech engine combining STT, intent recognition, and TTS for building voice agents. 9,400+ GitHub Stars.
Read more →
NVIDIA DeepStream 9.1 Multi-Camera 3D …
·2 min
NVIDIA DeepStream 9.1 Multi-Camera 3D Tracking: Principles, Architecture, and Applications
A deep dive into NVIDIA DeepStream 9.1 multi-camera 3D tracking: cross-camera Re-ID, 3D coordinate fusion, GPU-accelerated pipelines, and real-world deployments in retail, warehousing, and security.
Read more →
Apple M7 Ultra Leaked: Can 1.5TB Unifi…
·2 min
Apple M7 Ultra Leaked: Can 1.5TB Unified Memory Challenge NVIDIA Blackwell?
Apple's M7 Ultra chip rumored to feature 1.5TB unified memory targeting NVIDIA Blackwell-level AI performance. A deep dive into UMA advantages, what "matching Blackwell" really means, and Apple's AI strategy.
Read more →
Fine-Tuning Cosmos Models in One Day w…
·2 min
Fine-Tuning Cosmos Models in One Day with Agent Skills — Breaking 90% Accuracy
NVIDIA uses Autonomous Coding Agents and Agent Skills with TAO to fine-tune Cosmos visual reasoning models in one day, achieving over 90% accuracy.
Read more →
·3 min
CS Students' Summer Prep: How to Strategically Choose Between ML Internships and Data Analytics Roles
How should a CS+Stat junior efficiently prep for data/ML internships? We break down the real market gap, skill priorities, and a focused 3-month strategy.
Read more →
·3 min
OpenAI Removes the 5-Hour Limit: Intensifying AI Subscription Competition Makes Users the Biggest Winners
OpenAI removes the controversial 5-hour usage limit and raises subscription quotas. As competition with Anthropic, Google, and others intensifies, users are winning big in the AI arms race.
Read more →
4 Alternatives for Running CUDA on Non…
·3 min
4 Alternatives for Running CUDA on Non-NVIDIA Hardware: A Comparative Guide
A deep dive into running CUDA on non-NVIDIA hardware (AMD, Intel): comparing ROCm/HIP, ZLUDA, SYCL/oneAPI, and OpenCL across principles, use cases, and limitations.
Read more →
CUDA Kernel Fusion: A Practical Guide …
·2 min
CUDA Kernel Fusion: A Practical Guide to Reducing Memory Bandwidth and Launch Overhead
Learn how CUDA Kernel Fusion merges multiple GPU kernels to reduce global memory traffic and launch overhead, with real-world examples from AI inference and deep learning.
Read more →
How NVIDIA BioNeMo Breaks Through Co-F…
·2 min
How NVIDIA BioNeMo Breaks Through Co-Folding End-to-End Performance Bottlenecks
How NVIDIA BioNeMo Agent Toolkit uses agent-based orchestration to solve MSA preprocessing, pipeline scheduling, and end-to-end bottlenecks in OpenFold3 co-folding workloads for drug discovery.
Read more →
Computer Vision Career Paths: A Guide …
·3 min
Computer Vision Career Paths: A Guide for Undergraduates
Is Computer Vision worth pursuing as a career? This guide covers CV job market realities, master's vs. industry tradeoffs, edge deployment skills, and how to transition toward multimodal AI engineering.
Read more →
·2 min
Production LLM Deployment Guide: Model Selection, GPU Configuration, and Inference Engine Breakdown
A comprehensive decision framework for production LLM deployment: model selection (open-source vs. API), GPU VRAM configuration, and inference engine comparison including vLLM and TensorRT-LLM.
Read more →
·2 min
SentinelCV: An Open-Source AI Fall Detection Solution Using Your Existing Cameras
SentinelCV is an open-source YOLOv8-based fall detection system that works with existing CCTV and IP cameras. Get real-time alerts via Telegram — no new hardware needed.
Read more →
·3 min
Distributed AI Systems in Practice: A Complete Guide to Training, Inference, and Production Deployment
A deep dive into Distributed AI Systems: a new book distilling 10 years of AI engineering experience covering distributed training, inference optimization, and production model serving.
Read more →