#vLLM tensor parallelism

29 related articles

Jul 28, 2026·3 min

The Real Daily Life of MLE and MLOps Engineers: Key Differences Between Three Roles and Career Paths

An in-depth look at the real daily work of data scientists, MLEs, and MLOps engineers — covering responsibilities, essential tools, and career paths to help you find your direction in AI.

Jul 27, 2026·3 min

Kimi K3 Local Deployment Guide: Storage Requirements for q4 and q8 Quantization Explained

Detailed analysis of Kimi K3 quantization deployment options, comparing q4 vs q8 storage requirements, precision trade-offs, and hardware configurations for local self-hosting.

Jul 24, 2026·6 min

Hands-On with Panel AI On-Premises Deployment: One-Click Enterprise Adoption and Unified Compute Cluster Scheduling

In-depth review of Panel AI v1.1.1: second-level installation, no-public-IP networking, batch compute cluster management. Learn how enterprise AI on-premises deployment barriers are dramatically lowered.

Jul 22, 2026·3 min

V100 GPU Showdown: 4×16G PCIe vs. 2×32G SXM — Which Gives You More Bang for Your Buck?

Benchmarking 4×V100 16G PCIe vs. 2×V100 32G SXM adapter for local LLM inference. Prefill speed, decode speed, power limits, and bandwidth bottlenecks analyzed.

Jul 21, 2026·3 min

Claude Opus 5 Launch Imminent, Alibaba Qwen Officially Integrated into Apple Intelligence

Claude Opus 5 launches next week; Alibaba Qwen integrates into Apple Intelligence for Chinese users; 27B on-device model compressed to 3.8GB; open-source models narrow gap to closed-source by 3.3%.

Jul 17, 2026·2 min

vLLM Inference Framework Deep Dive: A Practical Guide to High-Concurrency DeepSeek Deployment

A deep dive into vLLM's five core features — KV cache management, continuous batching, and distributed inference — plus a full setup guide for deploying DeepSeek on a cloud server.

Jul 16, 2026·3 min

GPUStack in Action: Complete Guide to Day 0 Deployment of DeepSeek-V4-Flash

Deploy DeepSeek-V4-Flash DSpark on 8× H20-141G using GPUStack's SGLang backend on Day 0. Full walkthrough of Web UI config, parameter tuning, and 200 tokens/sec benchmark results.

Jul 16, 2026·2 min

ExLlamaV3 v1.0.0 Official Release: Faster Inference, Lossless KV Quantization, and Streamlined Dependencies

ExLlamaV3 v1.0.0 releases with lossless KV cache quantization via kernel fusion, removal of flash-attention-2/xformers, major GEMM/GEMV gains, and broader tensor parallelism support.

Jul 14, 2026·2 min

Production LLM Deployment Guide: Model Selection, GPU Configuration, and Inference Engine Breakdown

A comprehensive decision framework for production LLM deployment: model selection (open-source vs. API), GPU VRAM configuration, and inference engine comparison including vLLM and TensorRT-LLM.

Jul 13, 2026·2 min

Distributed AI Systems: A Practical Engineering Guide to Training, Inference, and Production Deployment

A deep dive into distributed AI systems engineering: data/model/tensor parallelism for training, KV cache, quantization, elastic scaling for inference, and cloud deployment with Kubernetes, Ray, and DeepSpeed.

Jul 13, 2026·3 min

Distributed AI Systems in Practice: A Complete Guide to Training, Inference, and Production Deployment

A deep dive into Distributed AI Systems: a new book distilling 10 years of AI engineering experience covering distributed training, inference optimization, and production model serving.

Jul 13, 2026·4 min

From Enter to First Token: Decrypting the Full LLM Inference Pipeline (Autoregression, KV Cache & Decoding Strategies)

From pressing Enter to the first character appearing, what happens inside an LLM? This article breaks down autoregressive generation, KV cache acceleration, and decoding strategies like temperature, Top-k, and Top-p.

Jul 12, 2026·5 min

The Five Unsolved Challenges of Production MLOps: A Deep Dive into GPU Scheduling, Fault-Tolerant Training, and Model Observability

A deep dive into the five genuinely tough challenges of production MLOps: fault-tolerant training on Spot instances, cross-team GPU scheduling, data reproducibility, model observability, and inference cost optimization.

Jul 12, 2026·3 min

SlickToken: A Deep Dive into GPU Fleet and Agentic Workflow Planning

SlickToken is a GPU fleet and agentic workflow planning tool for AI teams, supporting offline simulation, load testing, and capacity planning—no internet required to protect enterprise data.

Jul 11, 2026·6 min

A Deep Dive into the vLLM Inference Framework: Core Principles of Throughput Optimization and Interview Guide

An in-depth analysis of the vLLM inference framework's core principles: from the meaning of throughput (tokens/s), to the bottlenecks of autoregressive generation, to KV Cache, PagedAttention, and continuous batching.

Jul 11, 2026·6 min

GPT Live Full-Duplex, Grok 4.5 Coding, Seedream 5.0 Images: AI Accelerates Across Three Tracks Simultaneously

OpenAI's GPT Live full-duplex voice model, Grok 4.5 coding model with Cursor, and ByteDance's Seedream 5.0 Pro image generation launched together. A deep dive into three AI releases.

Jul 11, 2026·2 min

AI Agents for SGLang Kernel Optimization: Real-World Performance Gains and Engineering Methodology

SGLang's team converted expert knowledge into agent skills, achieving 71.4% throughput gains, TTFT reduced from 456ms to 168ms. A deep dive into agent-assisted kernel optimization methodology.

Jul 10, 2026·5 min

Real-World Comparison of 6 Open-Source LLMs for On-Premise Deployment: A Selection Guide to Avoid Pitfalls

A hands-on comparison of 6 open-source LLMs (DeepSeek, Qwen3, Zhipu GLM, Kimi K2, MiniMax M3, Tencent Hunyuan 3) for on-premise deployment—covering hardware cost, inference efficiency, and deployment difficulty.

Jul 9, 2026·7 min

LangChain Hands-On Guide: From LLM Calls to Agent Development

Master LangChain from scratch: the three limitations of LLMs, init_chat_model unified interface config, the Message type system, and the path from LLM calls to Agent development.

Jul 7, 2026·6 min

AMD MI355X Running GLM5.2: Inference Costs at Half of NVIDIA Blackwell

AMD MI355X achieves 2,626 tokens/sec throughput running GLM5.2 at half the TCO of NVIDIA Blackwell. Deep analysis of the technical logic, ROCm ecosystem progress, and market implications.