#inference speedup

106 related articles

Aug 1, 2026·1 min

The Third Dimension of Pretraining: A New Paradigm Claiming 6.2x Sample Efficiency and 250x Inference Speedup

Harvard and UIUC propose a third axis of pretraining, claiming 6.2x sample efficiency and 250x inference speedup. Deep analysis of this new paradigm's implications and key caveats.

Jul 25, 2026·3 min

D-Flash: The Secret Behind 3.5x LLM Inference Speedup with Diffusion Drafting

D-Flash uses fast diffusion parallel drafting and target feature KV injection to solve the autoregressive Drafter latency bottleneck in speculative decoding. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, surpassing EAGLE3 and MTP.

Jul 25, 2026·3 min

DeepSeek Open-Sources DeepSpec: A Full-Stack Speculative Decoding Library with 60–85% Inference Speedup

DeepSeek open-sources DeepSpec, a full-stack speculative decoding library delivering 60–85% inference speedup and up to 661% throughput gains in production. MIT license, 9 checkpoints, 9 benchmarks.

Jul 16, 2026·3 min

DeepSeek Open-Sources DSpark: 50%–400% LLM Inference Speedup Without Retraining

DeepSeek open-sources DSpark, delivering 50%–400% LLM inference speedup with no retraining or quantization, via semi-autoregressive drafting and confidence-scheduled verification.

Jul 7, 2026·6 min

14x Speedup: How Manticore Rebuilt the ONNX Embedding Inference Path

Manticore Search restructured its ONNX inference path to achieve 14x faster text embeddings. Deep dive into batching, session reuse, zero-copy memory, and thread tuning for vector search systems.

Jun 24, 2026·2 min

DiffusionGemma: Google's Text Diffusion Model Achieves 4x Inference Speedup

Google launches DiffusionGemma, a text diffusion language model achieving 4x faster inference than Gemma 4 series. Learn how text diffusion works and its impact on AI.

Aug 8, 2026·3 min

Software Optimization Makes a Single B200 Rival Dedicated Inference Chips

Analysis of how a single NVIDIA B200 GPU surpasses Groq LPU and approaches Cerebras performance through software optimization alone, covering CUDA kernels, TensorRT-LLM, and FP8 quantization.

Aug 7, 2026·3 min

picodl: Technical Analysis of a Deep Learning Library Built from Scratch with Pure NumPy

In-depth analysis of picodl, a lightweight deep learning library built from scratch with pure NumPy. Covers forward propagation, backpropagation, gradient computation, and discusses its educational value.

Aug 6, 2026·2 min

nanoAlphaZero: A Single-File JAX Implementation That Trains to 2700 Elo in 24 Hours

nanoAlphaZero is a single-file AlphaZero implementation in JAX that trains an Elo 2700+ chess model in 24 hours on a TPU v4-32. The entire RL pipeline is one JIT-compiled JAX function.

Aug 6, 2026·3 min

Unsloth Now Supports AMD GPUs: 2x Training Speedup and 70% VRAM Savings Across 500+ Models

Unsloth officially supports AMD GPUs across RDNA 3-4, Strix Halo, and MI300 series, delivering 2x training speedup and 70% VRAM savings on 500+ models with RL and vLLM weight sharing support.

Aug 6, 2026·3 min

The Real Efficiency Bottlenecks for AI Developers: It's Not the GPU — It's These Overlooked Areas

AI developers often think a bigger GPU will boost efficiency, but the real bottlenecks are often RAM, storage, networking, and workflow. Discover the overlooked upgrades that deliver the highest ROI.

Aug 5, 2026·3 min

Hadamard Transform: A Deep Dive into the Lightweight Deep Learning Technique That Replaces Multiplication with Addition and Subtraction

Deep dive into how the Hadamard Transform replaces matrix multiplication with only addition and subtraction for lightweight deep learning, covering FWHT principles, edge inference acceleration, and ultra-low-bit LLM quantization.

Aug 5, 2026·4 min

SAM 3 Auto-Labeling in Practice: Six Lessons Where Preprocessing Matters More Than the Model

Practical lessons from building a SAM 3 auto-labeling pipeline: vision embedding reuse, resolution handling, prompt engineering, threshold sweeping, and more.

Aug 4, 2026·2 min

Building Custom C/C++ Inference Engines: A Deep Dive into the Tradeoffs Between Performance, Control, and Engineering Cost

Deep analysis of why teams build custom C/C++ inference engines instead of using PyTorch or TensorRT, exploring performance, minimal dependencies, and long-term maintenance tradeoffs.

Aug 2, 2026·3 min

PyTorch vs TensorFlow: How Beginners Should Choose a Deep Learning Framework

Should deep learning beginners choose PyTorch or TensorFlow? This article compares both frameworks on research trends, ecosystem, and deployment, with practical switching advice.

Aug 1, 2026·3 min

How Cheap Can It Be to Serve a 2.8 Trillion Parameter Model? A Deep Dive into Inference Economics

Deep analysis of the real cost of serving a 2.8 trillion parameter model. From MoE sparse activation to batching scale effects and inference optimization, revealing why model size and serving cost are less correlated than assumed.

Jul 31, 2026·3 min

IBM Claims Quantum Advantage: Why the New Verification Technique Is a Key Breakthrough

IBM announces quantum advantage with a novel verification technique, addressing the trust problem in quantum computing results. Analysis of the method's logic, differences from Google's quantum supremacy, and implications for commercialization.

Jul 31, 2026·3 min

GPT-5.6 Slashes Prices by 80%: The Industry Logic Behind the AI Price War and Opportunities for Developers

OpenAI's GPT-5.6 series sees massive price cuts—Luna drops 80% to $0.20/M input tokens. Deep analysis of the AI price war's tech drivers, competitive landscape, and impact on developer costs and model selection.

Jul 31, 2026·3 min

GPT-5.6 Price Cut of 80%: The Industry Logic Behind the AI Price War and Opportunities for Developers

OpenAI's GPT-5.6 series sees major price cuts with Luna dropping 80% to $0.20/M input tokens. Analysis of the AI price war's technical drivers, competitive landscape, and impact on developer costs.

Jul 29, 2026·3 min

Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention

Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.