#LLM inference speedup
76 related articles
·4 min
Local Deployment of Claude Code: Principles and Practical Guide for Three Approaches
A detailed guide to locally deploying Claude Code with three approaches (LM Studio, Ollama, vLLM), covering architecture, protocol translation, hardware selection, and model recommendations.
Read more →
·3 min
vLLM Deep Dive: How PagedAttention Enables High-Throughput LLM Inference
Deep dive into vLLM's core technologies for high-throughput LLM inference, including PagedAttention memory management, continuous batching, distributed deployment, and comparisons with TensorRT-LLM.
Read more →
Deep Dives·3 min
Complete Guide to LLM Training: Pre-training, SFT Fine-tuning, and Preference Alignment Explained
Complete guide to the three core LLM training stages: pre-training, supervised fine-tuning (SFT), and preference alignment (DPO/PPO), covering LoRA, distillation, quantization, and pruning.
Read more →
Tech Frontiers·1 min
Qwen3.6 MTP Acceleration Benchmarked: Single GPU Inference Hits 220 token/s
Qwen3.6 experimental MTP-GGUF benchmarked: single GPU pushes 35B-A3B model to 220 token/s, 1.4x faster with zero accuracy loss. Covers MTP principles, optimal Draft Tokens strategy, and RTX 5090 results.
Read more →
Deep Dives·1 min
Qwen3.5 Deep Dive: Hybrid Attention Architecture Achieves 19x Speedup for Long Context
Deep analysis of Alibaba's open-source Qwen3.5 hybrid attention architecture, how Gated Delta Net achieves 19x speedup at 256K context, and multimodal results surpassing Gemini 3 Pro and GPT-5.2.
Read more →
Tutorials·4 min
NVIDIA Model Optimizer Post-Training Quantization (PTQ) Practical Guide
A deep dive into NVIDIA Model Optimizer's PTQ workflow, covering INT8/INT4 quantization principles, calibration methods, RTX GPU optimization, and best practices for deploying quantized LLMs on consumer GPUs.
Read more →
Deep Dives·5 min
Hugging Face Transformers: A Comprehensive Guide to the 160K-Star Open-Source AI Framework
Comprehensive guide to Hugging Face Transformers, the 160K-star GitHub framework—covering architecture, multimodal support, quantization, and inference optimization for loading, fine-tuning, and deploying pre-trained models.
Read more →
Deep Dives·3 min
Microsoft Tutel: A Deep Dive into the MoE Model Acceleration Library with FP4/FP8 Low-Precision Support
A deep dive into Microsoft's open-source Tutel MoE optimization library, supporting FP8, NVFP4, and MXFP4 multi-precision computation for DeepSeek, Kimi-K2, Qwen3, and other leading MoE models.
Read more →
Product Reviews·2 min
Unsloth: 80% VRAM Reduction — The Best Open-Source Tool for Local LLM Fine-Tuning
Unsloth is an open-source LLM training tool with 63K+ GitHub stars, supporting Gemma 4, Qwen 3, DeepSeek. Reduces VRAM by 50–80%, enabling RTX 4090 to fine-tune 7B models with a no-code Web UI.
Read more →
Research·3 min
SVDQuant: 4-bit Quantization Enables Efficient Diffusion Model Inference on Consumer GPUs
SVDQuant, an ICLR 2025 Spotlight paper, achieves 4-bit diffusion model quantization via low-rank decomposition that absorbs outliers, reducing memory by 75%. Open-source engine Nunchaku (3800+ stars) enables FLUX inference on consumer GPUs like RTX 4060.
Read more →
Tech Frontiers·2 min
DeepSeek V3.2 Released: Proprietary Sparse Attention (DSA) + 50% API Price Cut Explained
DeepSeek releases V3.2-Exp with proprietary DeepSeek Sparse Attention (DSA) for faster long-context training and inference, plus API prices cut over 50%.
Read more →
Tutorials·3 min
Unsloth Tutorial: Fine-Tune LLMs Locally with Multi-fold Speed Boost and 60% VRAM Savings
Unsloth is an open-source LLM fine-tuning tool with 63K GitHub stars, supporting Gemma 4, Qwen3, and DeepSeek. It achieves multi-fold training speedup and 60% VRAM reduction through kernel optimization, enabling fine-tuning on consumer GPUs.
Read more →
Tutorials·2 min
Unsloth Tutorial: A Guide to Fine-Tuning LLMs Locally with Reduced VRAM and Faster Training
Learn how Unsloth enables efficient local LLM fine-tuning with LoRA optimization, supporting Gemma 4, Qwen3, and DeepSeek while reducing VRAM usage by 50% and boosting training speed 2-5x.
Read more →
Product Reviews·5 min
Hugging Face Transformers: Deep Dive into the 160K-Star Open Source Framework
Deep dive into Hugging Face Transformers, covering core features, API design, model ecosystem, and practical code examples. Learn how this 160K-Star project lowers AI barriers and drives democratization across LLMs, computer vision, and multimodal AI.
Read more →
Tutorials·3 min
8000+ GitHub Stars: Deep Dive into the Most Comprehensive LLM Learning Resource Repository
Deep dive into the 8000+ star GitHub project awesome-LLM-resources, covering multimodal AI, AI Agents, MCP protocol, model training & inference, and more.
Read more →
Product Reviews·2 min
IBM Granite 4.1 Quantization Test: SVG Generation Comparison Across 21 GGUF Versions
IBM releases open-source Granite 4.1 models. Unsloth provides 21 GGUF quantized versions. Simon Willison's pelican SVG test reveals quantization level has minimal impact on 3B model output quality.
Read more →