#SGLang
6 related articles
SGLang v0.5.21 Released: PD Role Hot-Switching and Rust Prefix Cache
SGLang v0.5.21 is out with 779 PRs from 227 contributors, featuring PD role hot-switching, Rust prefix cache as default, new Decisions/Score APIs, and speculative decoding improvements.
SGLang v0.5.20 Released: High-Performance LLM Inference Engine Keeps Iterating
SGLang v0.5.20 is out. Learn about this high-performance LLM inference framework's core features, RadixAttention technology, and how to upgrade.
Industry InsightsSGLang Enters Finance: How AI Inference Infrastructure Is Reshaping Wall Street
SGLang co-hosts a finance AI inference event with Crusoe AI and Cloudflare, exploring LLM inference deployment in trading, risk management, and compliance — signaling Wall Street's shift to production-grade AI infrastructure.
Tech FrontiersCloudflare Contributes Critical KV Cache and Mooncake Fixes to SGLang
Cloudflare contributes decode KV cache offload and Mooncake recovery fixes to SGLang, resolving garbled output under high concurrency for Kimi K2.6 and enabling automatic fault recovery in distributed inference.
Tech FrontiersSGLang Hosts Agent Loops Office Hour, Focusing on Agentic Loop Architecture Optimization
SGLang team hosts an Agent Loops Office Hour exploring inference optimization for agentic loops, covering KV Cache reuse, low-latency multi-turn dialogue, and tool calling techniques.
TutorialsvLLM and SGLang Local Deployment Tutorial: A Practical Guide to 3-8x Performance Gains
Complete guide to deploying vLLM and SGLang locally. Compare performance vs LM Studio, deploy in 3 steps with Docker + AI assistant. Covers SGLang vs vLLM selection, 5090 VRAM optimization, and Cherry Studio integration.