#multi-GPU communication
63 related articles
Tutorials·2 min
NCCL Multi-GPU Communication Optimization in Practice: A Guide to Scaling HPC High-Performance Computing
Deep dive into NVIDIA NCCL multi-GPU communication library principles and optimization strategies, covering AllReduce, NVLink, and GPUDirect RDMA to help HPC and AI developers master scaling from single-node to massive clusters.
Read more →
·3 min
Software Optimization Makes a Single B200 Rival Dedicated Inference Chips
Analysis of how a single NVIDIA B200 GPU surpasses Groq LPU and approaches Cerebras performance through software optimization alone, covering CUDA kernels, TensorRT-LLM, and FP8 quantization.
Read more →
·3 min
Feasibility Analysis: Deploying DeepSeek V4 Flash on Dual DGX Spark Systems
In-depth feasibility analysis of deploying DeepSeek V4 Flash on two NVIDIA DGX Spark units offline, examining memory bandwidth, MoE communication overhead, and quantization strategies.
Read more →
·3 min
GPU Utilization at 100% Doesn't Mean Full Load: How to Truly Diagnose Training Bottlenecks
nvidia-smi showing 100% GPU utilization doesn't mean optimal training efficiency. Learn about DCGM, PyTorch Profiler, and MFU metrics for diagnosing real GPU training bottlenecks.
Read more →
·2 min
Local GPU or Cloud for Machine Learning? A Purchasing Decision Guide
Should ML beginners buy a local GPU laptop or use cloud computing? This guide analyzes cloud platforms like Colab and Kaggle vs. gaming laptops, offering budget-friendly recommendations and hybrid strategies.
Read more →
·3 min
persistent-inference: Solving TF/Keras Cold Start Problems with Just Two Files
Deep dive into the persistent-inference open-source project: solve TF/Keras cold start problems with just two files by keeping models resident in memory, eliminating reload overhead.
Read more →
·3 min
Dual RTX 3060 Running DeepSeek V4 Flash: IQ2_M Quantized Inference Benchmarked at 3.5 tok/s
Benchmarking DeepSeek V4 Flash on dual RTX 3060 GPUs with 96GB RAM at IQ2_M quantization achieving 3.5 tokens/sec. Covers hardware choices, 2-bit quantization techniques, and local LLM deployment optimization.
Read more →
·4 min
Multi-Purpose Home Server Build Guide: Streaming + AI Inference + App Hosting on One Machine
How to build a $500 multi-purpose home server for Jellyfin streaming, Ollama local AI inference, web app hosting, and Pi-hole ad blocking with dual RTX 3060 GPUs.
Read more →
·3 min
Getting Started with MARL (Multi-Agent Reinforcement Learning): A Complete Path from Theory to Code
A systematic guide to learning MARL from theory to code, covering CleanRL, PettingZoo, PyMARL tools, IQL/VDN/QMIX/MADDPG algorithm progression, and practical tips for bridging theory and implementation.
Read more →
·2 min
Hygon Launches 512-Thread CPU and AI GPU, Taking Aim at Intel Xeon and Nvidia
Hygon unveils a 512-thread server CPU and AI GPU, challenging Intel Xeon and Nvidia. Analysis of specs, applications, ecosystem challenges, and strategic significance.
Read more →
·2 min
Hygon Launches 512-Thread CPU and AI GPU, Taking Aim at Intel Xeon and Nvidia
Hygon unveils a 512-thread server CPU and AI GPU, directly challenging Intel Xeon and Nvidia. A deep dive into specs, applications, ecosystem challenges, and strategic significance.
Read more →
·3 min
Production-Grade LLM Inference Deployment: A Practical K8s Handbook
Complete guide to deploying production-grade LLM inference on Kubernetes, covering GPU scheduling, vLLM engine selection, autoscaling, observability, and cost optimization.
Read more →
·3 min
ncnn Vulkan Cross-Platform Edge Inference in Practice: Solving the GPU Fragmentation Deployment Challenge
Facing GPU fragmentation on edge devices, the PostSlate team used ncnn's Vulkan backend for cross-platform ML inference, achieving 10× speedup on RTX 4070 with half the model size and zero runtime installation.
Read more →
·3 min
AMD CDNA5 Architecture Deep Dive: Technical Evolution and the AI Computing Competition Landscape
Deep analysis of AMD's CDNA5 architecture covering Chiplet packaging upgrades, HBM memory evolution, and low-precision compute optimization, examining how AMD challenges NVIDIA's AI chip dominance.
Read more →
·3 min
Kimi K3 Local Deployment Guide: Storage Requirements for q4 and q8 Quantization Explained
Detailed analysis of Kimi K3 quantization deployment options, comparing q4 vs q8 storage requirements, precision trade-offs, and hardware configurations for local self-hosting.
Read more →
·3 min
DeepSeek's In-House AI Chip: A Critical Step Toward Breaking China's NVIDIA Dependence
DeepSeek is reportedly developing its own AI chip, moving from algorithms to hardware to achieve software-hardware co-optimization. An in-depth analysis of its strategic rationale, key challenges, and implications for China's AI industry autonomy.
Read more →
·7 min
AI Daily: T-Head Open-Sources SAIL Software Stack at WAIC, Flurry of Large Model Releases
T-Head open-sources AI software stack T-Head SAIL at WAIC to lower the barrier for domestic chip development; Kimi K3 tops the WebDev leaderboard; Qwen 3.8 Max Preview cuts prices aggressively; Moonshot prepares a Hong Kong IPO; and Oracle switches its data center to a fuel cell microgrid.
Read more →
Block Low-Rank Compression: A Guide to…
·2 min
Block Low-Rank Compression: A Guide to Accelerating Large Model Inference on Memory-Constrained GPUs
Learn how Block Low-Rank (BLR) decomposition compresses large model memory usage and accelerates GPU inference, including CUDA kernel optimization and combination with quantization and pruning.
Read more →
·2 min
vLLM Inference Framework Deep Dive: A Practical Guide to High-Concurrency DeepSeek Deployment
A deep dive into vLLM's five core features — KV cache management, continuous batching, and distributed inference — plus a full setup guide for deploying DeepSeek on a cloud server.
Read more →
·3 min
GPUStack in Action: Complete Guide to Day 0 Deployment of DeepSeek-V4-Flash
Deploy DeepSeek-V4-Flash DSpark on 8× H20-141G using GPUStack's SGLang backend on Day 0. Full walkthrough of Web UI config, parameter tuning, and 200 tokens/sec benchmark results.
Read more →