#vLLM inference

286 related articles

Jul 27, 2026·3 min

DeepSeek-OCR Deployment & Fine-Tuning in Practice: From vLLM Inference to Unsloth Fine-Tuning

Complete guide to DeepSeek-OCR from vLLM inference deployment and Unsloth model loading to fine-tuning, covering cloud server setup, GPU selection, and code examples — all on a single 4090 GPU.

Jul 17, 2026·2 min

vLLM Inference Framework Deep Dive: A Practical Guide to High-Concurrency DeepSeek Deployment

A deep dive into vLLM's five core features — KV cache management, continuous batching, and distributed inference — plus a full setup guide for deploying DeepSeek on a cloud server.

Jul 11, 2026·6 min

A Deep Dive into the vLLM Inference Framework: Core Principles of Throughput Optimization and Interview Guide

An in-depth analysis of the vLLM inference framework's core principles: from the meaning of throughput (tokens/s), to the bottlenecks of autoregressive generation, to KV Cache, PagedAttention, and continuous batching.

Jun 6, 2026·3 min

vLLM Deep Dive: How PagedAttention Enables High-Throughput LLM Inference

Deep dive into vLLM's core technologies for high-throughput LLM inference, including PagedAttention memory management, continuous batching, distributed deployment, and comparisons with TensorRT-LLM.

Aug 1, 2026·2 min

A Slow-Mode Vision for Codex: Doubling Coding Quota with Flex API

A developer proposes a Flex API-based slow mode for Codex, trading speed for nearly double the usage quota. We analyze the product logic, technical feasibility, and business challenges.

Aug 1, 2026·3 min

How Cheap Can It Be to Serve a 2.8 Trillion Parameter Model? A Deep Dive into Inference Economics

Deep analysis of the real cost of serving a 2.8 trillion parameter model. From MoE sparse activation to batching scale effects and inference optimization, revealing why model size and serving cost are less correlated than assumed.

Aug 1, 2026·3 min

Self-Hosting a Code Review AI Agent: From Architecture Design to Engineering Implementation

A deep dive into building and self-hosting a code review AI Agent from scratch, covering architecture design, context management, model selection, and noise control.

Aug 1, 2026·4 min

Heretic Uncensoring Technology Explained: Practical Applications on Jamba2 and Qwen3.5 Open-Source Models

Deep dive into Heretic uncensoring technology applied to Jamba2-Mini, Qwen3.5-9B, and 27B open-source models, exploring how refusal rates dropped from 97% to 4% and the safety debates involved.

Aug 1, 2026·1 min

Predictive KV Replication: Breaking Through the Bursty Traffic Bottleneck in LLM Inference

Deep dive into predictive speculative KV replication: how anticipating bursty traffic and pre-replicating KV Cache can reduce LLM inference tail latency.

Jul 31, 2026·3 min

GPT-5.6 Luna Review: Outperforms Google's Flagship at a Lower Price

GPT-5.6 Luna tops Google's flagship on the Artificial Analysis Intelligence Index while priced below Google's entry-level model. A deep dive into what this performance-cost breakthrough means.

Jul 31, 2026·2 min

DeepSeek V4 Flash Weights Open-Sourced: A New Option for Lightweight LLM Deployment

DeepSeek V4 Flash model weights reportedly open-sourced. This article analyzes its lightweight positioning, open-weight value, comparisons with closed-source models, and deployment guidance.

Jul 31, 2026·3 min

GPT-5.6 Slashes Prices by 80%: The Industry Logic Behind the AI Price War and Opportunities for Developers

OpenAI's GPT-5.6 series sees massive price cuts—Luna drops 80% to $0.20/M input tokens. Deep analysis of the AI price war's tech drivers, competitive landscape, and impact on developer costs and model selection.

Jul 31, 2026·3 min

GPT-5.6 Price Cut of 80%: The Industry Logic Behind the AI Price War and Opportunities for Developers

OpenAI's GPT-5.6 series sees major price cuts with Luna dropping 80% to $0.20/M input tokens. Analysis of the AI price war's technical drivers, competitive landscape, and impact on developer costs.

Jul 30, 2026·3 min

GPT-5.6 Price Cut by 80%: The Deeper Game Behind the AI Model Price War

OpenAI announces GPT-5.6 Luna 80% cheaper, Terra 20% cheaper, targeting Gemini Flash. Analysis of the AI price war's business logic, Google's moat, Microsoft's hidden gains, and open-source impact.

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Why memory bandwidth (GB/s), not VRAM size, determines local LLM inference speed. Includes tokens/sec formula, GPU bandwidth comparison, and a practical card selection framework.

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Learn why memory bandwidth (GB/s)—not VRAM size—determines local LLM inference speed. Get the tokens/sec formula, GPU bandwidth comparisons, and a practical card selection hierarchy.

Jul 30, 2026·3 min

Building a Streaming LLM Backend in Go: Deep Dive into the SDK and React Frontend Library

Deep dive into an open-source Go SDK for building streaming LLM backends, covering streaming responses, tool-calling architecture, and companion React library for end-to-end integration.

Jul 30, 2026·2 min

Gemini Suddenly Appeared in Gmail Then Disappeared? A Deep Dive into Gradual Rollout Mechanisms

A Reddit user found Gemini features in Gmail without a subscription, only to lose access a day later. This article explains Google's gradual rollout strategy and AI feature deployment logic.

Jul 30, 2026·3 min

Is LLM Inference Really That Expensive? Breaking Down the Cost Reduction Logic of GPU Compute and Inference Frameworks

Analyzing real LLM inference costs: from B200 GPU compute gains, vLLM framework optimization to MTP multi-token prediction, explaining why serving costs are widely overestimated.

Jul 30, 2026·2 min

Behind the Open-Source Model Frenzy: Who Will Provide Cheap Inference Services?

Open-source LLM weights don't equal low-cost access for developers. This article analyzes the inference service gap in open-source AI and how providers like Together AI and Groq are addressing it.