#SGLang deployment
48 related articles
Tech Frontiers·2 min
LFM2.5-8B-A1B: A MoE Model with 1.5B Active Parameters Delivering 4x Its Weight Class Performance
Liquid AI releases LFM2.5-8B-A1B, a MoE model with 8B total params but only 1.5B active, matching 6B-class models in tool calling. Supports 128K context, local deployment, multilingual, with SGLang Day-0 support.
Read more →
Industry Insights·2 min
SGLang Enters Finance: How AI Inference Infrastructure Is Reshaping Wall Street
SGLang co-hosts a finance AI inference event with Crusoe AI and Cloudflare, exploring LLM inference deployment in trading, risk management, and compliance — signaling Wall Street's shift to production-grade AI infrastructure.
Read more →
Industry Insights·2 min
AMD MI355X Beats B200: Full-Stack Optimization Breakdown for 5% Lower TCO on DeepSeek-R1 Inference
AMD Instinct MI355X achieves 5% lower TCO than NVIDIA B200 on DeepSeek-R1 disaggregated inference via SGLang+MoRI full-stack optimization with 1.25x per-GPU throughput.
Read more →
Tech Frontiers·1 min
Cloudflare Contributes Critical KV Cache and Mooncake Fixes to SGLang
Cloudflare contributes decode KV cache offload and Mooncake recovery fixes to SGLang, resolving garbled output under high concurrency for Kimi K2.6 and enabling automatic fault recovery in distributed inference.
Read more →
Industry Insights·3 min
Deep Dive into Three Major LLM Career Paths: Requirements, Tech Stacks, and Career Prospects
Deep analysis of three core LLM roles—Application Engineer, Development Engineer, and Algorithm Engineer—covering technical requirements, salary thresholds, and career prospects including RAG, fine-tuning, and inference deployment.
Read more →
Tutorials·3 min
OpenAI Open-Sources GPT-OSS: Run O4-Level Models on 16GB VRAM — Complete Deployment Guide
OpenAI open-sources GPT-OSS (20B/120B) with MOE architecture and native FP4 precision. Run O3-level reasoning on a single RTX 4090. Full deployment guide for Ollama, vLLM, and more.
Read more →
Deep Dives·3 min
Microsoft Tutel: A Deep Dive into the MoE Model Acceleration Library with FP4/FP8 Low-Precision Support
A deep dive into Microsoft's open-source Tutel MoE optimization library, supporting FP8, NVFP4, and MXFP4 multi-precision computation for DeepSeek, Kimi-K2, Qwen3, and other leading MoE models.
Read more →
Tech Frontiers·2 min
Kimi K2 Official Verification Tool K2-Vendor-Verifier: One-Click API Vendor Model Accuracy Detection
Moonshot AI open-sources K2-Vendor-Verifier to verify third-party Kimi K2 API vendor inference accuracy. Learn how this tool helps developers detect over-quantization, model substitution, and other API market risks.
Read more →