#LLM inference optimization

792 related articles

Jul 11, 2026·6 min

A Deep Dive into the vLLM Inference Framework: Core Principles of Throughput Optimization and Interview Guide

An in-depth analysis of the vLLM inference framework's core principles: from the meaning of throughput (tokens/s), to the bottlenecks of autoregressive generation, to KV Cache, PagedAttention, and continuous batching.

Jun 22, 2026·4 min

KV Cache Saves 20x on Costs: The Underlying Principles and Practical Tips for LLM Inference Optimization

Deep dive into how KV Cache reduces LLM API costs by 20x. From Transformer attention matrix multiplication overhead to prompt caching best practices, understand the fundamentals of AI inference cost optimization.

Jun 13, 2026·2 min

A Complete Guide to LLM Infrastructure: Core Challenges from GPU Clusters to Inference Optimization

A deep dive into core challenges and key technologies for LLM infrastructure, covering GPU cluster management, inference optimization, distributed training, cost control, and observability.

Jul 28, 2026·3 min

Claude Code vs Codex: Hands-On Comparison and Installation Guide for AI Programming Tools

In-depth comparison of Claude Code and Codex AI programming tools covering accuracy, installation, and network setup tips to help developers choose the best solution.

Jul 28, 2026·4 min

Statistical Foundations of Machine Learning: From MLE to EWMA and Deep Learning Optimizer Principles

Deep dive into core ML statistics: MLE derivations, multivariate Gaussian, linear regression and least squares equivalence, empirical risk minimization, method of moments, and how EWMA connects to Adam optimizer.

Jul 28, 2026·4 min

Cursor Start India Plan at ₹649/Month: Decoding the Regional Pricing Strategy for AI Coding Tools

Cursor launches its Start plan for India at ₹649/month (~$7.70), featuring Grok 4.5 and Composer Agent. A deep dive into the PPP pricing strategy and its market implications.

Jul 28, 2026·3 min

Spring AI Explained: Unified API, RAG, and Enterprise AI Development Guide

Deep dive into Spring AI framework's core features including provider-agnostic unified API abstraction, RAG retrieval-augmented generation, and structured output to help Java developers build enterprise AI apps.

Jul 28, 2026·3 min

Spring AI Explained: Unified API, RAG, and Enterprise AI Development Practical Guide

Jul 28, 2026·3 min

Agent Skill Routing: Engineering Trade-offs Between Retrieval and LLM, and the Two-Stage Layered Architecture

Deep dive into Agent skill routing: comparing pure model vs. pure retrieval approaches, with a detailed two-stage layered architecture balancing accuracy, latency, and cost.

Jul 28, 2026·3 min

Kimi K3 Deep Dive: How a 3 Trillion Parameter Open-Weight Model Reaches the Frontier

Deep dive into Kimi K3: the largest open-weight model at 3 trillion parameters, surpassing Opus-level models in Agentic coding with 896-expert MoE architecture, 1M token context, at Sonnet pricing.

Jul 28, 2026·3 min

Deep Dive into Inkling: The First Frontier Open-Source Multimodal LLM from the West

Thinking Machines releases Inkling, an open-source multimodal LLM with near-trillion MoE parameters, 1M token context, Apache 2.0 license. Deep dive into architecture, benchmarks, and pricing.

Jul 28, 2026·4 min

Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop

Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.

Jul 28, 2026·4 min

Laguna S 2.1 Deep Dive: Hands-On Local Deployment of a 118B Parameter Coding Model

In-depth review of Poolside's Laguna S 2.1 open-source coding model: MoE architecture, RL training, DGX Spark local deployment, and real-world agentic coding tests with 8B active parameters.

Jul 28, 2026·3 min

Anthropic Opus 5 Real-World Testing: Outperforms Competitors at Half the Price

Anthropic Opus 5 hands-on review: first to break 30% on ARC-AGI, near Fable 5 agentic coding at half the price. Benchmarks, token costs, and GPT-5.6 comparison.

Jul 28, 2026·3 min

Anthropic Opus 5 Real-World Testing: Outperforms Competitors at Half the Price

Anthropic Opus 5 hands-on review: first to break 30% on ARC-AGI, near Fable 5 in agentic coding at half the price. Benchmarks, token costs, and GPT-5.6 comparison.

Jul 28, 2026·5 min

Running GPT-5.6 Inside Claude Code: Best Practices for Subagent Orchestration and System Prompts

Deep dive into running OpenAI GPT-5.6 inside Claude Code: comparing Codex vs Claude Code on subagent orchestration, workflow design, and system prompt quality, revealing how harness engineering determines model output.

Jul 28, 2026·4 min

Kimi K3 Deep Dive: 2.8 Trillion Parameter Open-Source Model Closes the Gap with Closed-Source Frontier for the First Time

Moonshot AI releases Kimi K3 open-weight model with 2.8T parameters and 1M token context. Our deep dive covers coding, 3D dev, agent capabilities, and safety concerns.

Jul 28, 2026·4 min

Kimi K2/K3 Technical Speculation: Deep Dive into RL Hyperparameters and Multi-Teacher Distillation

Deep analysis of RL hyperparameter tuning challenges and 9-policy multi-teacher distillation in Kimi K2/K3 training, exploring the shift from scale to training craft.

Jul 28, 2026·3 min

A 9B Model Beats Frontier LLMs After Just $500 in RL Fine-Tuning

Just $500 in RL fine-tuning enables a 9B open-source model to outperform frontier LLMs on catalog review tasks. Analysis of when small-model RL works and its enterprise implications.

Jul 28, 2026·2 min

Gemini Always Crashes on the 16th Turn? Error 1076 Root Cause Analysis and Workarounds

Google's Gemini consistently triggers Error 1076 on the 16th conversation turn, regardless of context size. Analysis points to a session state management defect, with three workarounds provided.