#tensor parallelism

84 related articles

Jul 17, 2026·2 min

vLLM Inference Framework Deep Dive: A Practical Guide to High-Concurrency DeepSeek Deployment

A deep dive into vLLM's five core features — KV cache management, continuous batching, and distributed inference — plus a full setup guide for deploying DeepSeek on a cloud server.

CUDA 13.3 Adds Carryless Multiplicatio…

Jul 17, 2026·2 min

CUDA 13.3 Adds Carryless Multiplication: A Key Step Forward for GPU Cryptography Acceleration

CUDA 13.3 introduces native carryless multiplication support, closing a 15-year GPU gap in AES-GCM, CRC, and cryptographic acceleration. Here's what it means.

Building RL-Powered Autonomous Researc…

Jul 17, 2026·2 min

Building RL-Powered Autonomous Research Agents with NVIDIA NeMo

How NVIDIA NeMo combines reinforcement learning to train agent skills and build an Autoresearch workflow capable of autonomously running ML experiments end-to-end.

Apple M7 Ultra Leaked: Can 1.5TB Unifi…

Jul 16, 2026·2 min

Apple M7 Ultra Leaked: Can 1.5TB Unified Memory Challenge NVIDIA Blackwell?

Apple's M7 Ultra chip rumored to feature 1.5TB unified memory targeting NVIDIA Blackwell-level AI performance. A deep dive into UMA advantages, what "matching Blackwell" really means, and Apple's AI strategy.

Jul 16, 2026·3 min

GPUStack in Action: Complete Guide to Day 0 Deployment of DeepSeek-V4-Flash

Deploy DeepSeek-V4-Flash DSpark on 8× H20-141G using GPUStack's SGLang backend on Day 0. Full walkthrough of Web UI config, parameter tuning, and 200 tokens/sec benchmark results.

Jul 16, 2026·3 min

LongCat 2.0: Meituan Trains a 1.6 Trillion Parameter LLM on Domestic Chips

Meituan open-sources LongCat 2.0, a 1.6T-parameter MoE model trained on 50,000+ custom chips without NVIDIA GPUs or Google TPUs, rivaling OpenAI and Google.

Jul 16, 2026·2 min

ExLlamaV3 v1.0.0 Official Release: Faster Inference, Lossless KV Quantization, and Streamlined Dependencies

ExLlamaV3 v1.0.0 releases with lossless KV cache quantization via kernel fusion, removal of flash-attention-2/xformers, major GEMM/GEMV gains, and broader tensor parallelism support.

CUDA Kernel Fusion: A Practical Guide …

Jul 15, 2026·2 min

CUDA Kernel Fusion: A Practical Guide to Reducing Memory Bandwidth and Launch Overhead

Learn how CUDA Kernel Fusion merges multiple GPU kernels to reduce global memory traffic and launch overhead, with real-world examples from AI inference and deep learning.

How NVIDIA BioNeMo Breaks Through Co-F…

Jul 15, 2026·2 min

How NVIDIA BioNeMo Breaks Through Co-Folding End-to-End Performance Bottlenecks

How NVIDIA BioNeMo Agent Toolkit uses agent-based orchestration to solve MSA preprocessing, pipeline scheduling, and end-to-end bottlenecks in OpenFold3 co-folding workloads for drug discovery.

Best Laptops for AI/ML Students: A Dee…

Jul 14, 2026·2 min

Best Laptops for AI/ML Students: A Deep Dive Comparing Three Popular Models

Lenovo LOQ, HP Omen, or MacBook Air M5? A deep dive comparing GPU performance, RAM, and CUDA compatibility to help AI/ML students find the right laptop.

Jul 14, 2026·2 min

Production LLM Deployment Guide: Model Selection, GPU Configuration, and Inference Engine Breakdown

A comprehensive decision framework for production LLM deployment: model selection (open-source vs. API), GPU VRAM configuration, and inference engine comparison including vLLM and TensorRT-LLM.

JAX Host Offloading: A Practical Guide…

Jul 14, 2026·2 min

JAX Host Offloading: A Practical Guide to Breaking Through LLM Training Memory Bottlenecks

Memory capacity is the core bottleneck in LLM training. This guide explores JAX-based host offloading — covering optimizer state offloading, activation strategies, PCIe bandwidth trade-offs, and how it complements activation recomputation.

Jul 13, 2026·2 min

Distributed AI Systems: A Practical Engineering Guide to Training, Inference, and Production Deployment

A deep dive into distributed AI systems engineering: data/model/tensor parallelism for training, KV cache, quantization, elastic scaling for inference, and cloud deployment with Kubernetes, Ray, and DeepSpeed.

Jul 13, 2026·3 min

Distributed AI Systems in Practice: A Complete Guide to Training, Inference, and Production Deployment

A deep dive into Distributed AI Systems: a new book distilling 10 years of AI engineering experience covering distributed training, inference optimization, and production model serving.

Jul 13, 2026·3 min

The Rise of Diffusion Language Models: DiffusionGemma Crushes Autoregressive Models in Speed Tests

Diffusion language model DiffusionGemma dramatically outpaces autoregressive Deepseek Flash in speed tests. Explore the tech behind diffusion vs. autoregressive models and their challenges.

Jul 13, 2026·4 min

From Enter to First Token: Decrypting the Full LLM Inference Pipeline (Autoregression, KV Cache & Decoding Strategies)

From pressing Enter to the first character appearing, what happens inside an LLM? This article breaks down autoregressive generation, KV cache acceleration, and decoding strategies like temperature, Top-k, and Top-p.

Jul 12, 2026·3 min

Programs as Data Flow: Rethinking Architecture Paradigms Through Building a 3D Editor in C++

A developer builds a 3D editor from scratch in C++, modeling programs as dataflow graphs rather than object collections. Exploring dataflow vs. OOP, Greenspun's Tenth Rule, and the debugging dividend of visual programming.

Jul 12, 2026·5 min

The Five Unsolved Challenges of Production MLOps: A Deep Dive into GPU Scheduling, Fault-Tolerant Training, and Model Observability

A deep dive into the five genuinely tough challenges of production MLOps: fault-tolerant training on Spot instances, cross-team GPU scheduling, data reproducibility, model observability, and inference cost optimization.

Mesh LLM: A Practical Exploration of B…

Jul 12, 2026·3 min

Mesh LLM: A Practical Exploration of Building a P2P Distributed AI Inference Network with iroh

Mesh LLM leverages the Rust P2P framework iroh to integrate compute from scattered nodes, exploring a viable path for decentralized LLM inference. This article analyzes its architecture, challenges, and prospects.

Jul 12, 2026·3 min

SlickToken: A Deep Dive into GPU Fleet and Agentic Workflow Planning

SlickToken is a GPU fleet and agentic workflow planning tool for AI teams, supporting offline simulation, load testing, and capacity planning—no internet required to protect enterprise data.