#SGLang

44 related articles

Jul 14, 2026·2 min

QuantaMind: An Open-Source Tool for Testing AI Agent Reliability Locally

QuantaMind is a free, open-source local AI Agent reliability testing tool using pass^k scoring and deterministic evaluation, supporting Ollama, llama.cpp, vLLM, and more.

Jul 13, 2026·4 min

From Enter to First Token: Decrypting the Full LLM Inference Pipeline (Autoregression, KV Cache & Decoding Strategies)

From pressing Enter to the first character appearing, what happens inside an LLM? This article breaks down autoregressive generation, KV cache acceleration, and decoding strategies like temperature, Top-k, and Top-p.

Jul 12, 2026·3 min

DeepSeek-V4 Flash RL Training Successfully Ported to AMD MI355X GPU: A Complete Breakdown

The Miles team and AMD announce the full port of DeepSeek-V4 Flash RL training to AMD Instinct MI355X GPUs on ROCm, boosting AIME pass@1 from 0.39 to 0.49—a milestone for compute ecosystem diversity.

Jul 11, 2026·6 min

A Deep Dive into the vLLM Inference Framework: Core Principles of Throughput Optimization and Interview Guide

An in-depth analysis of the vLLM inference framework's core principles: from the meaning of throughput (tokens/s), to the bottlenecks of autoregressive generation, to KV Cache, PagedAttention, and continuous batching.

Jul 11, 2026·2 min

MOSS-Transcribe-Diarize: A 0.9B End-to-End Multi-Speaker Speech Transcription Model

MosiAI open-sources MOSS-Transcribe-Diarize-0.9B: unified speech transcription and speaker diarization, 128K context for 90-min audio, hotword boosting, SGLang Day-0 support, edge-deployable.

Jul 11, 2026·3 min

The Software Compounding Effect: The Real Deciding Factor in AI Infrastructure Competition

Beyond the GPU arms race, software is becoming AI infrastructure's true compounding engine. A deep dive into optimization stacking, the CUDA ecosystem moat, and how open source scales individual breakthroughs into community wealth.

Jul 11, 2026·2 min

Deep Dive into Tencent Hunyuan Hy3: How a 295B MoE Model Cracks the Agent Deployment Challenge

Tencent Hunyuan Hy3 launches with a 295B MoE architecture activating just 21B params and 256K context. Hallucination cut from 12.5% to 5.4%, MRCR nearly doubled, with MTP and EAGLE decoding and Day-0 SGLang support.

Jul 11, 2026·5 min

Ollama Raises $65M Series B: 85% of Fortune 500 Have Already Deployed Local LLMs

Local LLM tool Ollama closes a $65M Series B, bringing total funding to $88M. With 9M developers and 85% of Fortune 500 having deployed internally, this deep dive explores why enterprises embrace local LLMs: compliance, Agent cost savings, and open-source ecosystem.

Jul 11, 2026·5 min

GPT-5.6 Trio Released: A Deep Dive into Sol/Terra/Luna and the Breakthrough in Design Judgment

OpenAI officially releases the GPT-5.6 series with three models: flagship Sol, balanced Terra, and economy Luna. A deep dive into its core breakthroughs—a step change in design judgment and enhanced computer-use capabilities.

Jul 10, 2026·2 min

Tencent Hunyuan V3 Open-Sourced: Full Breakdown of the 295B MoE Architecture — 256K Context Is Its Biggest Weakness

Tencent open-sources Hunyuan V3: 295B MoE model with 21B active params, claims to beat DeepSeek-V4-Pro. Supports FP8, vLLM/SGLang — but 256K context and no multimodal are key drawbacks.

Jun 24, 2026·4 min

Kimi K2.7 Code Deep Dive: The Cost-Performance Battle of a Trillion-Parameter Coding Model

Deep dive into Moonshot AI's Kimi K2.7 Code: MoE architecture details, benchmark analysis, API pricing vs Claude/GPT, 6x speed version, and practical guidance for developers evaluating adoption.

Jun 21, 2026·2 min

Baseten Raises $1.5 Billion: Why AI Inference Infrastructure Has Become a Capital Darling

AI inference startup Baseten is raising $1.5B at a $130B valuation. We analyze why inference infrastructure is booming, the competitive landscape, and what this mega-round signals.

Jun 20, 2026·4 min

Codewell Hands-On Review: Is This Terminal AI Coding Assistant with Nearly 40K GitHub Stars Actually Worth It?

Hands-on review of Codewell (formerly DeepSeek2E), the open-source terminal AI coding assistant with nearly 40K GitHub stars. Supports 25 LLM providers, local models at zero cost, and MIT license.

Jun 14, 2026·3 min

5 Ways to Deploy LLMs Locally: From Getting Started to Production

A complete guide to 5 local LLM deployment methods: LlamaCPP, Ollama, LM Studio, vLLM/SGLang, and MLX-LM — from personal dev to production environments.

Manus Hands-On Review: How Does This AI Agent Perform on the DeepSeek Tech Stack?

Product Reviews

Jun 3, 2026·3 min

Manus Hands-On Review: How Does This AI Agent Perform on the DeepSeek Tech Stack?

Hands-on review of Manus AI Agent on the DeepSeek tech stack, analyzing task execution, Chinese reasoning capabilities, strengths, limitations, and the potential of domestic LLMs in Agent applications.

DeepSeek-V3.2 Released: Coding and Math Capabilities Join the Global Top Tier

Tech Frontiers

Jun 3, 2026·2 min

DeepSeek-V3.2 Released: Coding and Math Capabilities Join the Global Top Tier

DeepSeek-V3.2 released with coding, math, and Agent capabilities matching Gemini 3.0 Pro, setting new open-source SOTA. Detailed analysis of performance gains, use cases, and deployment tips.

llama.cpp MTP Acceleration Deployment Guide: Configuration Steps & Real-World Benchmarks

Tutorials

Jun 2, 2026·3 min

llama.cpp MTP Acceleration Deployment Guide: Configuration Steps & Real-World Benchmarks

Guide to enabling MTP multi-Token prediction acceleration in llama.cpp, covering CUDA setup, desktop configuration, model selection, and benchmarks showing ~60 Token/s with Qwen3 27B.

Research

May 30, 2026·2 min

Agent Loops in Practice: Transforming Token Output into Productivity from CUDA Kernels to Automated Research

Deep dive into how the Humanize framework transforms LLM tokens into engineering productivity via Agent Loops. Covers KDA winning CUDA kernel contests, virtual hardware optimization, and 50% research cost reduction.

Step 3.7 Flash: Deep Dive into the 198B Sparse MoE Multimodal Model

Tech Frontiers

May 30, 2026·2 min

Step 3.7 Flash: Deep Dive into the 198B Sparse MoE Multimodal Model

Deep dive into StepFun AI's Step 3.7 Flash, a 198B sparse MoE vision-language model with 256K context and 3-level reasoning, excelling in multimodal understanding, AI coding, and Agent tool orchestration.

LFM2.5-8B-A1B: A MoE Model with 1.5B Active Parameters Delivering 4x Its Weight Class Performance

Tech Frontiers

May 30, 2026·2 min

LFM2.5-8B-A1B: A MoE Model with 1.5B Active Parameters Delivering 4x Its Weight Class Performance

Liquid AI releases LFM2.5-8B-A1B, a MoE model with 8B total params but only 1.5B active, matching 6B-class models in tool calling. Supports 128K context, local deployment, multilingual, with SGLang Day-0 support.