#high concurrency
840 related articles
·2 min
vLLM Inference Framework Deep Dive: A Practical Guide to High-Concurrency DeepSeek Deployment
A deep dive into vLLM's five core features — KV cache management, continuous batching, and distributed inference — plus a full setup guide for deploying DeepSeek on a cloud server.
Read more →
·3 min
PostgreSQL Lock Contention: The Root Cause and Solutions for High-Concurrency Scalability Bottlenecks
An in-depth analysis of PostgreSQL's lock scalability bottlenecks under high concurrency: LWLock contention, the cost of the multi-process model, negative scaling, and practical strategies like connection pooling, transaction optimization, and horizontal scaling.
Read more →
·2 min
SGLang Integrates DSpark: How Confidence-Driven Speculative Decoding Cracks the High-Concurrency Failure Problem
SGLang officially integrates DSpark, solving the core pain point of speculative decoding failure under high-concurrency batches via confidence-driven variable-length verification. Supports Qwen3 and DeepSeek-V4, hitting 383.7 tok/s on B300.
Read more →
·3 min
Designing MPMC Queues: Balancing Bounded Waiting and High-Performance Concurrency
An in-depth look at the core design challenges of MPMC queues: bounded waiting, lock-free mechanisms, sequence-number slots, dual-path strategies, and helping. Helping engineers balance throughput, fairness, and latency.
Read more →
Product Reviews·1 min
Xiaozhi ESP32 Backend Rewritten in Go: Building a High-Concurrency AI Voice Interaction Service
Deep dive into xiaozhi-esp32-server-golang: a Go rewrite of the Xiaozhi ESP32 backend with WebSocket/MQTT, voiceprint recognition, MCP calls & more.
Read more →
·2 min
Aquifer Open Source: A Traffic Smoothing Solution for Peak Shaving Bursty GPU Inference Workloads
Aquifer is an open-source traffic smoothing runtime that uses durable queue buffering and backend backpressure to solve burst traffic challenges in GPU inference services, enabling peak shaving and cost reduction.
Read more →
·4 min
6 Must-Do Tasks Before Deploying Enterprise AI Agents: A Complete Guide to Observability + Evaluation
A detailed guide to 6 critical engineering challenges for enterprise AI Agents before production, covering Langfuse-based tracing, observability, evaluation stages, prompt governance, and high-concurrency architecture.
Read more →
·2 min
Struggling to Choose a Gemini Model? Analyzing UX Pain Points in the Multi-Model AI Era
Facing Gemini Pro, Flash, Ultra and many variants, users often suffer choice paralysis. This article analyzes AI model naming confusion, information asymmetry, and proposes UX solutions like smart defaults and intent-first design.
Read more →
·3 min
CVE-2026-53361 Vulnerability Analysis: Linux Kernel AF_UNIX Container Escape Mechanism and Defenses
In-depth analysis of CVE-2026-53361 Linux kernel vulnerability: AF_UNIX socket GC and MSG_PEEK race condition causing UAF, container escape exploitation chain, and defense strategies.
Read more →
·3 min
The Overlooked "Boring" Infrastructure Behind Production-Grade AI Agents
Exploring the core challenges of AI Agents moving from demo to production: idempotency, approval states, retries, action ledgers, audit tables, and other critical infrastructure design patterns.
Read more →
·3 min
How to Become a Stronger Engineer in the AI Era: Neither Surrendering Nor Rejecting
How can engineers avoid skill atrophy from over-relying on AI coding tools? This article provides an actionable growth path covering system design, debugging, and code review to build core competitiveness.
Read more →
·3 min
Heym: Deep Dive into a Self-Hosted Visual Agent Building Platform
Heym is a self-hostable visual platform for building agentic systems, offering multi-agent workflow orchestration, built-in observability, human approval mechanisms, and MCP protocol support.
Read more →
·4 min
The Creator of Claude Code Shares 10 High-Efficiency AI Coding Tips
Claude Code creator Boris shares 10 practical AI coding tips in his viral talk, covering setup, prompting, planning, team collaboration, SDK usage, and multi-instance concurrency.
Read more →
·3 min
GitHub Daily · August 12: Claude Code Ecosystem Explosion and Extreme Edge Model Compression
GitHub Trending Aug 12: Claude Code ecosystem explodes with diagram-design topping charts, needle compresses models to 14MB for edge AI, and Rust rises in AI infrastructure.
Read more →
·3 min
Running Local LLMs on 24GB VRAM? Your Actual Usable Memory Is Less Than You Think
A detailed breakdown of actual usable VRAM when running local LLMs on 24GB GPUs. Covers the three memory buckets — model weights, KV cache, and runtime headroom — with structured planning methods.
Read more →
·3 min
Full-Stack LangChain Application Development: Architecture Selection and Practical Guide
A detailed guide on full-stack LangChain architecture design, covering FastAPI backend setup, streaming responses, React frontend integration, and practical tool selection with LangServe and LangGraph.
Read more →
·2 min
Voice Agent Latency Optimization: Why Averages Are Lying to You
Deep analysis of three voice AI Agent latency pitfalls: averages hiding tail latency, pipeline jitter stacking, and regional differences. Practical P95/P99 measurement and end-to-end optimization tips.
Read more →
·2 min
A Go Developer's Guide to Gemini: Complete Model Family Breakdown
A comprehensive Gemini model family guide for Go developers, covering Pro vs Flash selection strategies, multimodal capabilities, official Go SDK integration, and token management practices.
Read more →
·2 min
Open-Source LLM Stock Analysis System: A Zero-Cost Automated Intelligent Investment Assistant
Deep dive into the trending GitHub project daily_stock_analysis: an LLM-powered multi-market stock analysis system with real-time news sentiment analysis, decision dashboards, and zero-cost automated scheduling for individual investors.
Read more →
·3 min
Uncle Bob's Swarm-Forge: A Minimalist Framework for Orchestrating Multi-AI Agents with Clojure
Uncle Bob open-sources swarm-forge, a lightweight multi-AI agent coordination tool built with Clojure. Explore its design philosophy, Clojure's concurrency advantages, and its significance in the multi-agent framework ecosystem.
Read more →