#distributed LLM inference
507 related articles
Tutorials·4 min
Agentic RAG Practical Guide: The Complete Evolution from Traditional RAG to Agent-Based Retrieval
Deep dive into Agentic RAG vs traditional RAG, with ChatPDF case study and LangChain code walkthrough covering tool design, multi-turn iteration, and autonomous decision-making for LLM engineers.
Read more →
Deep Dives·3 min
Microsoft Tutel: A Deep Dive into the MoE Model Acceleration Library with FP4/FP8 Low-Precision Support
A deep dive into Microsoft's open-source Tutel MoE optimization library, supporting FP8, NVFP4, and MXFP4 multi-precision computation for DeepSeek, Kimi-K2, Qwen3, and other leading MoE models.
Read more →
Tutorials·3 min
4-bit QLoRA Fine-tuning LLaMA 3 in Practice: A Guide to Training 8 Billion Parameter LLMs on Consumer GPUs
Learn how to fine-tune LLaMA 3 8B on consumer GPUs using QLoRA and Unsloth. Covers 4-bit quantization, LoRA adaptation, Alpaca data format, training pipeline setup, and model deployment—runs on RTX 3090.
Read more →
Product Reviews·3 min
awesome-LLM-resources: A Comprehensive Analysis of the 8,200-Star LLM Learning Resource Repository on GitHub
In-depth analysis of the 8,200-star GitHub project awesome-LLM-resources, covering multimodal generation, Agents, model training, MCP protocol, and more — a one-stop LLM learning guide.
Read more →
Tutorials·2 min
Cursor 3.3 Context Usage Analysis: A Complete Guide to Understanding and Optimization
Cursor 3.3 adds Agent context usage analysis, helping developers visualize Rules, MCP, Skills context consumption. Learn how to optimize AI programming efficiency.
Read more →
Tech Frontiers·2 min
Cursor Launches Claude Opus 4 Fast Mode: 2.5x Faster, 6x More Expensive — Is It Worth It?
Cursor launches Claude Opus 4 Fast Mode with 2.5x speed but 6x cost. We analyze use cases, cost tradeoffs, and practical tips to help you decide if it's worth it.
Read more →
Deep Dives·4 min
Hugging Face Transformers: Deep Dive into the 160K-Star Open Source AI Framework
Deep dive into Hugging Face Transformers: the 160K-Star open-source framework covering Pipeline API, Auto Classes, multi-modal models, and the full HF ecosystem for AI inference and training.
Read more →
Product Reviews·3 min
Cube Studio: A Comprehensive Analysis of Tencent's Open-Source Cloud-Native AI Platform
Deep dive into Tencent Music's open-source Cube Studio cloud-native AI platform, covering distributed training, LLM fine-tuning & inference, Pipeline orchestration, and domestic hardware adaptation.
Read more →
Tutorials·3 min
GitHub 8K Stars: Deep Dive into the Most Comprehensive LLM Resource Repository
Deep analysis of the GitHub project awesome-LLM-resources with 8,200+ Stars, covering multimodal AI, Agents, MCP protocol, model training, inference optimization, and coding assistants.
Read more →
Product Reviews·2 min
Unsloth: 80% VRAM Reduction — The Best Open-Source Tool for Local LLM Fine-Tuning
Unsloth is an open-source LLM training tool with 63K+ GitHub stars, supporting Gemma 4, Qwen 3, DeepSeek. Reduces VRAM by 50–80%, enabling RTX 4090 to fine-tune 7B models with a no-code Web UI.
Read more →
Product Reviews·4 min
ComfyUI-Copilot: An AI Assistant for Building Stable Diffusion Workflows with Natural Language
ComfyUI-Copilot is an open-source AI assistant plugin supporting natural language workflow building, smart node recommendations, and parameter optimization. With 5,100+ GitHub stars, it dramatically lowers the barrier to Stable Diffusion workflow creation.
Read more →
Product Reviews·3 min
awesome-LLM-resources: A Deep Dive into GitHub's 8K-Star Comprehensive LLM Learning Resource Repository
Deep dive into awesome-LLM-resources, a GitHub 8K-star project covering multimodal AI, AI Agents, MCP protocol, model training/inference, and AI coding tools — a one-stop LLM learning guide.
Read more →
Tech Frontiers·3 min
LangGraph Multi-Agent Company Research Tool: A Practical Guide to Open-Source AI Due Diligence
Deep dive into the open-source company-research-agent: LangGraph multi-agent architecture + Tavily search + dual-LLM collaboration for automated company due diligence and competitive intelligence.
Read more →
Deep Dives·3 min
LLM 0.32 Refactor: How Message Sequences and Streaming Chunks Are Reshaping Python LLM Interactions
Simon Willison releases LLM 0.32a0 alpha with message sequences and typed streaming chunks, redefining how Python developers interact with LLMs while maintaining full backward compatibility.
Read more →
Deep Dives·3 min
Augment Code Architecture Revealed: Dedicated Subagents Replace KV Cache, Cutting Costs by 90%
Deep dive into how Augment Code uses Mercury 2 dedicated subagents to replace traditional KV cache, achieving 82% faster context compaction, 90% lower summarization costs, and 30% reduced LLM spending.
Read more →
Product Reviews·3 min
Tensorlake Skills: Giving AI Coding Agents Real-Time Access to Up-to-Date Documentation
Tensorlake Skills is a skill module for AI coding agents like Claude Code and OpenAI Codex, routing them to real-time docs on sandboxes, orchestration, and SDKs to fix outdated training data issues.
Read more →
Deep Dives·3 min
Why Is AI Agent Testing So Hard? How Simulation Testing Cracks the Infinite Input Space Problem
AI Agents face infinite input spaces and non-deterministic outputs. Learn how simulation testing systematically validates Agent reliability through scenario generation, environment simulation, and behavior evaluation.
Read more →
Tech Frontiers·2 min
DeepSeek-V4-Pro API at 75% Off: Full Breakdown of Million-Token Context & Mainstream Tool Integrations
DeepSeek-V4-Pro API is 75% off through May 5, 2026. With 1M-token context and Claude Code/OpenCode integration, here's the full pricing and developer guide.
Read more →
Product Reviews·2 min
Opik: An Open-Source LLM Observability Platform for Debugging, Evaluation, and Monitoring All in One Place
Deep dive into Opik: an open-source platform for LLM app lifecycle management with tracing, automated evaluation, hallucination detection, and production monitoring for RAG and Agent workflows.
Read more →
Product Reviews·2 min
Opik: Open-Source LLM Observability Platform for Debugging, Evaluation, and Monitoring All in One
Deep dive into Opik: an open-source platform for LLM app lifecycle management with tracing, automated evaluation, hallucination detection, and production monitoring. 19K+ GitHub stars.
Read more →