#KV cache bandwidth
182 related articles
OpenAI's First Custom AI Chip Jalapeño…
·2 min
OpenAI's First Custom AI Chip Jalapeño: A Deep Dive into Its Full-Stack AI Platform Strategy
OpenAI unveils Jalapeño, its first custom AI chip built with Broadcom, optimized for LLM inference. A deep dive into its architecture, strategy, and impact on NVIDIA and the AI chip landscape.
Read more →
·2 min
Minimax at $40/Month for 35 Billion Tokens: A Deep Dive into This Fable Alternative
Minimax offers 35 billion tokens for just $40/month (~$1.14 per million tokens), far below mainstream AI pricing. Compare Minimax vs Fable for the best value AI inference solution.
Read more →
·4 min
KV Cache Saves 20x on Costs: The Underlying Principles and Practical Tips for LLM Inference Optimization
Deep dive into how KV Cache reduces LLM API costs by 20x. From Transformer attention matrix multiplication overhead to prompt caching best practices, understand the fundamentals of AI inference cost optimization.
Read more →
·3 min
Spring AI + Ollama: Run Large Language Models Locally at Zero Cost
Learn how to integrate Spring AI with Ollama to run open-source LLMs like Llama and Gemma locally for free. Covers setup, configuration, and code — switch from OpenAI by just changing dependencies.
Read more →
·2 min
MiniMax M3 Launches on Fireworks: 512K Context and MSA Sparse Attention Explained
MiniMax M3 launches on Fireworks with 512K context and multimodal input. MSA sparse attention delivers 9x prefill and 15x decode speedups. Deep dive into architecture, pricing, and open-model competition.
Read more →
·3 min
Fireworks AI Launches Qwen 3.7 Plus: Zero Data Retention and 99.9% SLA for Enterprise Deployment
Fireworks AI launches Qwen 3.7 Plus with latency/throughput optimization, zero data retention, and 99.9% SLA enterprise guarantees. Explore the full-stack deployment solution for commercial open-source model inference.
Read more →
·4 min
Local Deployment of Claude Code: Principles and Practical Guide for Three Approaches
A detailed guide to locally deploying Claude Code with three approaches (LM Studio, Ollama, vLLM), covering architecture, protocol translation, hardware selection, and model recommendations.
Read more →
·3 min
Complete Guide to Alibaba Cloud Website Architecture: The Full Request Path from DNS to Auto Scaling
A systematic guide to Alibaba Cloud website architecture covering DNS, CDN, WAF, CLB/ALB, ECS, Redis, NAS/OSS, and auto scaling along the full user request path.
Read more →
·3 min
5 Ways to Deploy LLMs Locally: From Getting Started to Production
A complete guide to 5 local LLM deployment methods: LlamaCPP, Ollama, LM Studio, vLLM/SGLang, and MLX-LM — from personal dev to production environments.
Read more →
·2 min
DiffusionGemma: Google's Open-Source Diffusion Language Model Exceeding 500 Tokens/s
Google releases DiffusionGemma, an open-source diffusion language model with Apache 2.0 license. The 26B-parameter MoE model achieves over 500 tokens/s in real-world tests.
Read more →
·2 min
A Complete Guide to LLM Infrastructure: Core Challenges from GPU Clusters to Inference Optimization
A deep dive into core challenges and key technologies for LLM infrastructure, covering GPU cluster management, inference optimization, distributed training, cost control, and observability.
Read more →
·3 min
Claude Code Local Deployment: A Practical Guide to Connecting Local LLMs
Complete guide to deploying Claude Code locally with Ollama, LM Studio, or vLLM. Covers architecture, protocol translation, hardware requirements, and model selection for zero-cost, private AI coding.
Read more →
·4 min
Cursor Composer2 Training Revealed: A Complete Guide to Distributed Reinforcement Learning Engineering
Deep dive into how Cursor trained Composer2: two-stage architecture, global distributed clusters, MOE numerical alignment, simulation anti-cheating, and more.
Read more →
·3 min
Hands-On Testing of DS4 Engine by Redis Creator: How Does DeepSeek V4 Perform Locally on a 128GB Mac?
Redis creator Antirez's DS4 inference engine tested: running DeepSeek V4 Flash locally on a 128GB Mac via asymmetric structure-aware quantization, with real-world coding benchmarks.
Read more →
·3 min
vLLM Deep Dive: How PagedAttention Enables High-Throughput LLM Inference
Deep dive into vLLM's core technologies for high-throughput LLM inference, including PagedAttention memory management, continuous batching, distributed deployment, and comparisons with TensorRT-LLM.
Read more →
Industry Insights·3 min
China's Internet Giants Collectively Expand AI Capital Expenditure: Six Key Beneficiary Sectors in the Computing Infrastructure Supply Chain
China's internet giants collectively increase AI CapEx as computing infrastructure shifts from expectations to delivery. Analysis of six key beneficiary sectors including AI data centers, chips, and storage.
Read more →
Tutorials·3 min
DeepSeek Multi-Agent Matrix + UE5.8 Official MCP: A Collaborative Development Workflow in Practice
A complete workflow for collaborative UE5 development using DeepSeek multi-Agent matrix and UE5.8 official MCP, covering pure C++ architecture, agent roles, cache optimization, and automated code review.
Read more →
Deep Dives·4 min
Core Principles of the Transformer Architecture: A Deep Dive into Self-Attention Mechanisms and Engineering Optimizations
Deep dive into Transformer architecture covering self-attention QKV mechanics, Encoder-Decoder structure, Flash Attention memory optimization, RoPE positional encoding, and GQA inference acceleration.
Read more →
Tutorials·2 min
Tutorial: Deploying a PD-Disaggregated SGLang Multi-Node Inference Cluster on AMD GPUs
Learn how to deploy a PD-disaggregated SGLang inference cluster on AMD GPUs using a single config file, boosting LLM throughput and latency performance.
Read more →
Tech Frontiers·2 min
SGLang v0.5.12.post1 Released: DeepSeek V4 Stability Fixes and Blackwell Adaptation
SGLang v0.5.12.post1 stability patch details: 12 critical fixes covering DeepSeek V4 garbled text and crashes, NIXL PD disaggregated inference logic, Blackwell B300 adaptation, and cold start optimization.
Read more →