#distributed LLM inference
526 related articles
Tutorials·3 min
Spring AI + MCP Protocol in Practice: Tool Calling, Security Authentication, and Enterprise Deployment
Deep dive into Spring AI and MCP protocol integration, covering tool calling, OAuth security, horizontal scaling, and context optimization for enterprise AI Agent services.
Read more →
Tutorials·2 min
Tutorial: Deploying a PD-Disaggregated SGLang Multi-Node Inference Cluster on AMD GPUs
Learn how to deploy a PD-disaggregated SGLang inference cluster on AMD GPUs using a single config file, boosting LLM throughput and latency performance.
Read more →
Tech Frontiers·2 min
SGLang v0.5.12.post1 Released: DeepSeek V4 Stability Fixes and Blackwell Adaptation
SGLang v0.5.12.post1 stability patch details: 12 critical fixes covering DeepSeek V4 garbled text and crashes, NIXL PD disaggregated inference logic, Blackwell B300 adaptation, and cold start optimization.
Read more →
Industry Insights·2 min
AMD MI355X Beats B200: Full-Stack Optimization Breakdown for 5% Lower TCO on DeepSeek-R1 Inference
AMD Instinct MI355X achieves 5% lower TCO than NVIDIA B200 on DeepSeek-R1 disaggregated inference via SGLang+MoRI full-stack optimization with 1.25x per-GPU throughput.
Read more →
Tech Frontiers·1 min
Cloudflare Contributes Critical KV Cache and Mooncake Fixes to SGLang
Cloudflare contributes decode KV cache offload and Mooncake recovery fixes to SGLang, resolving garbled output under high concurrency for Kimi K2.6 and enabling automatic fault recovery in distributed inference.
Read more →
Tech Frontiers·1 min
SGLang Hosts Agent Loops Office Hour, Focusing on Agentic Loop Architecture Optimization
SGLang team hosts an Agent Loops Office Hour exploring inference optimization for agentic loops, covering KV Cache reuse, low-latency multi-turn dialogue, and tool calling techniques.
Read more →
Industry Insights·3 min
Deep Dive into Three Major LLM Career Paths: Requirements, Tech Stacks, and Career Prospects
Deep analysis of three core LLM roles—Application Engineer, Development Engineer, and Algorithm Engineer—covering technical requirements, salary thresholds, and career prospects including RAG, fine-tuning, and inference deployment.
Read more →
Tech Frontiers·2 min
Meta Muse Spark Released: A Comprehensive Analysis of the Native Multimodal Reasoning Model
Meta Superintelligence Labs releases Muse Spark, a native multimodal reasoning model supporting visual chain of thought, tool-use, and multi-agent orchestration. Deep dive into its capabilities and competitive positioning.
Read more →
Industry Insights·2 min
Major Overhaul of GitHub's Bug Bounty Program: Higher Quality Thresholds and Redefined Shared Responsibility Boundaries
GitHub overhauls its Bug Bounty program with higher quality standards, clearer shared responsibility boundaries, and adjusted low-risk vulnerability rewards. A deep dive into the three core changes.
Read more →
Tutorials·3 min
Cursor Multi-Agent Workflow: A Practical Guide to Main Thread & Grunt Collaboration
Learn the Cursor "Main Thread & Grunt" multi-agent workflow: use a high-tier model for complex tasks and a low-tier model for simple tasks in parallel to maximize AI coding efficiency.
Read more →
Industry Insights·3 min
How Jane Street Built a Custom AI Programming Toolchain for OCaml
Jane Street's AI team details how they built a custom LLM toolchain for OCaml, covering workspace snapshot training data, RL with code evaluation, and the AID editor architecture.
Read more →
Deep Dives·3 min
Claude Sub-agents vs Agent Teams: How to Choose Between Two AI Multi-Agent Collaboration Patterns
Deep comparison of Claude sub-agents vs agent teams: architecture differences, use cases, and real-world results. A Pokémon RPG case study shows how agent teams achieve 95%+ feature completeness.
Read more →
Tutorials·2 min
Claude Code Sub-Agent: A Practical Guide to Multi-Agent Collaboration for Enhanced Development Efficiency
Deep dive into Claude Code Sub-Agent mechanism with a practical blog writing + Git commit case study, showing how multi-agent collaboration solves instruction loss and context bloat issues.
Read more →
Tutorials·3 min
Open-Source Multi-Agent Intelligent Diagnostic System: Deep Dive into MCP + RAG + Skill Routing Architecture
Deep dive into an open-source multi-Agent diagnostic system built on modified OneCall, featuring MCP real-time interaction, RAG-enhanced Q&A, and Skill routing to minimize Token consumption.
Read more →
Tutorials·3 min
Agent Skills Decision Trees: Enabling Autonomous Decision-Making for AI Coding Assistants
Learn how to embed decision tree logic in Agent Skills to give AI coding assistants like Antigravity and Claude Code autonomous decision-making. Includes a Code Review Router case study with complexity scoring and failover.
Read more →
Tutorials·3 min
Getting Started with AI Full-Stack Development: A Knowledge Framework from Machine Learning to Large Language Models
A systematic guide to the relationships between AI, machine learning, deep learning, and large language models, helping developers build a clear knowledge framework and find an efficient learning path.
Read more →
Tutorials·2 min
Huawei Hermes Agent Manual Explained: Five-Layer Memory Architecture and Multi-Agent Collaboration in Practice
Deep dive into Huawei's 100-page Hermes Agent manual: five-layer memory architecture solving AI amnesia, self-evolution loops for continuous optimization, and multi-agent collaboration engineering.
Read more →
Industry Insights·2 min
NVIDIA Blackwell Sets New STAC-AI Records for Financial LLM Inference
NVIDIA Blackwell GPU sets new LLM inference records in STAC-AI financial benchmark. Explore Blackwell architecture advantages, TensorRT-LLM co-optimization, and LLM applications in trading and risk management.
Read more →
Tech Frontiers·2 min
AI Code Review in Practice: Why Machines Are Better Suited for PR Reviews Than Humans
Explore how AI code review solves PR review inefficiency in dev teams. Analysis of AI's advantages, use cases, tool comparisons, and human-AI collaboration best practices.
Read more →
Tech Frontiers·1 min
Devin Launches Multi-Instance Parallel Mode: AI Programming Evolves from Solo Operations to Army-Scale Collaboration
Devin launches multi-instance parallel mode, letting developers dispatch multiple AI Agents for different tasks simultaneously. Analysis of its impact on development efficiency and how multi-Agent collaboration redefines the developer role.
Read more →