#RadixAttention
17 related articles
·6 min
Hands-On with Panel AI On-Premises Deployment: One-Click Enterprise Adoption and Unified Compute Cluster Scheduling
In-depth review of Panel AI v1.1.1: second-level installation, no-public-IP networking, batch compute cluster management. Learn how enterprise AI on-premises deployment barriers are dramatically lowered.
Read more →
·3 min
GPUStack in Action: Complete Guide to Day 0 Deployment of DeepSeek-V4-Flash
Deploy DeepSeek-V4-Flash DSpark on 8× H20-141G using GPUStack's SGLang backend on Day 0. Full walkthrough of Web UI config, parameter tuning, and 200 tokens/sec benchmark results.
Read more →
·4 min
LingBot-World 2.0 Meets SGLang: A Deep Dive into the Evolution of Real-Time Interactive World Models
SGLang-Diffusion now officially supports LingBot-World 2.0, delivering leaps in resolution and temporal consistency. With live sessions, chunked streaming, and camera control, world models achieve low-latency controllable interaction.
Read more →
·2 min
MOSS-Transcribe-Diarize: A 0.9B End-to-End Multi-Speaker Speech Transcription Model
MosiAI open-sources MOSS-Transcribe-Diarize-0.9B: unified speech transcription and speaker diarization, 128K context for 90-min audio, hotword boosting, SGLang Day-0 support, edge-deployable.
Read more →
·3 min
TB/s Memory Tier Accelerates SGLang: How X-Mem Reduces TTFT by 6.7x
Netpreme integrates X-Mem™ MPU into SGLang HiCache, achieving up to 6.7× TTFT reduction and 33–50% TPS gains at 98% prefix cache hit rates. Here's the technical breakdown.
Read more →
·2 min
Deep Dive into Tencent Hunyuan Hy3: How a 295B MoE Model Cracks the Agent Deployment Challenge
Tencent Hunyuan Hy3 launches with a 295B MoE architecture activating just 21B params and 256K context. Hallucination cut from 12.5% to 5.4%, MRCR nearly doubled, with MTP and EAGLE decoding and Day-0 SGLang support.
Read more →
·2 min
AI Agents for SGLang Kernel Optimization: Real-World Performance Gains and Engineering Methodology
SGLang's team converted expert knowledge into agent skills, achieving 71.4% throughput gains, TTFT reduced from 456ms to 168ms. A deep dive into agent-assisted kernel optimization methodology.
Read more →
·5 min
Ollama Raises $65M Series B: 85% of Fortune 500 Have Already Deployed Local LLMs
Local LLM tool Ollama closes a $65M Series B, bringing total funding to $88M. With 9M developers and 85% of Fortune 500 having deployed internally, this deep dive explores why enterprises embrace local LLMs: compliance, Agent cost savings, and open-source ecosystem.
Read more →
·3 min
5 Ways to Deploy LLMs Locally: From Getting Started to Production
A complete guide to 5 local LLM deployment methods: LlamaCPP, Ollama, LM Studio, vLLM/SGLang, and MLX-LM — from personal dev to production environments.
Read more →
Research·2 min
Agent Loops in Practice: Transforming Token Output into Productivity from CUDA Kernels to Automated Research
Deep dive into how the Humanize framework transforms LLM tokens into engineering productivity via Agent Loops. Covers KDA winning CUDA kernel contests, virtual hardware optimization, and 50% research cost reduction.
Read more →
Tech Frontiers·2 min
SGLang v0.5.12.post1 Released: DeepSeek V4 Stability Fixes and Blackwell Adaptation
SGLang v0.5.12.post1 stability patch details: 12 critical fixes covering DeepSeek V4 garbled text and crashes, NIXL PD disaggregated inference logic, Blackwell B300 adaptation, and cold start optimization.
Read more →
Tech Frontiers·2 min
Step 3.7 Flash: Deep Dive into the 198B Sparse MoE Multimodal Model
Deep dive into StepFun AI's Step 3.7 Flash, a 198B sparse MoE vision-language model with 256K context and 3-level reasoning, excelling in multimodal understanding, AI coding, and Agent tool orchestration.
Read more →
Tech Frontiers·2 min
LFM2.5-8B-A1B: A MoE Model with 1.5B Active Parameters Delivering 4x Its Weight Class Performance
Liquid AI releases LFM2.5-8B-A1B, a MoE model with 8B total params but only 1.5B active, matching 6B-class models in tool calling. Supports 128K context, local deployment, multilingual, with SGLang Day-0 support.
Read more →
Industry Insights·2 min
SGLang Enters Finance: How AI Inference Infrastructure Is Reshaping Wall Street
SGLang co-hosts a finance AI inference event with Crusoe AI and Cloudflare, exploring LLM inference deployment in trading, risk management, and compliance — signaling Wall Street's shift to production-grade AI infrastructure.
Read more →
Industry Insights·2 min
AMD MI355X Beats B200: Full-Stack Optimization Breakdown for 5% Lower TCO on DeepSeek-R1 Inference
AMD Instinct MI355X achieves 5% lower TCO than NVIDIA B200 on DeepSeek-R1 disaggregated inference via SGLang+MoRI full-stack optimization with 1.25x per-GPU throughput.
Read more →
Tech Frontiers·1 min
Cloudflare Contributes Critical KV Cache and Mooncake Fixes to SGLang
Cloudflare contributes decode KV cache offload and Mooncake recovery fixes to SGLang, resolving garbled output under high concurrency for Kimi K2.6 and enabling automatic fault recovery in distributed inference.
Read more →
Tech Frontiers·1 min
SGLang Hosts Agent Loops Office Hour, Focusing on Agentic Loop Architecture Optimization
SGLang team hosts an Agent Loops Office Hour exploring inference optimization for agentic loops, covering KV Cache reuse, low-latency multi-turn dialogue, and tool calling techniques.
Read more →