#LLM inference speed

885 related articles

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Why memory bandwidth (GB/s), not VRAM size, determines local LLM inference speed. Includes tokens/sec formula, GPU bandwidth comparison, and a practical card selection framework.

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Learn why memory bandwidth (GB/s)—not VRAM size—determines local LLM inference speed. Get the tokens/sec formula, GPU bandwidth comparisons, and a practical card selection hierarchy.

Jul 28, 2026·3 min

Apple Silicon LLM Inference Speed Benchmarks: A Complete Guide to Local Large Model Performance on Mac

In-depth analysis of Apple Silicon local LLM inference speed benchmarks covering M-series memory bandwidth, model quantization, MLX framework optimization, and Mac configuration guidance.

Jul 25, 2026·3 min

D-Flash: The Secret Behind 3.5x LLM Inference Speedup with Diffusion Drafting

D-Flash uses fast diffusion parallel drafting and target feature KV injection to solve the autoregressive Drafter latency bottleneck in speculative decoding. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, surpassing EAGLE3 and MTP.

Jul 17, 2026·4 min

DSpark Open Source: DeepSeek's Inference Acceleration Tech That Doubles LLM Speed

DeepSeek and Peking University open-source DSpark, an inference acceleration technology using semi-autoregressive architecture and dynamic scheduling to boost LLM speed by 50%+ and double GPU concurrency without quality loss.

Jul 16, 2026·3 min

DeepSeek Open-Sources DSpark: 50%–400% LLM Inference Speedup Without Retraining

DeepSeek open-sources DSpark, delivering 50%–400% LLM inference speedup with no retraining or quantization, via semi-autoregressive drafting and confidence-scheduled verification.

Aug 1, 2026·3 min

DeepSeek V4 Flash Released: Performance Approaching Claude at Just $0.18 per Million Tokens

DeepSeek V4 Flash launches with benchmark scores approaching Claude Opus 4.8 at just $0.18 per million output tokens. Deep analysis of performance, pricing, and industry impact.

Aug 1, 2026·3 min

Ollama + Qwen-Agent: Building a Local Private AI Assistant — From Deployment to Security Best Practices

A complete guide to building a local private AI assistant with Ollama and Qwen-Agent. Covers RAG knowledge integration, voice interaction, and permission isolation for a secure local AI Agent architecture.

Aug 1, 2026·2 min

A Slow-Mode Vision for Codex: Doubling Coding Quota with Flex API

A developer proposes a Flex API-based slow mode for Codex, trading speed for nearly double the usage quota. We analyze the product logic, technical feasibility, and business challenges.

Aug 1, 2026·3 min

How Cheap Can It Be to Serve a 2.8 Trillion Parameter Model? A Deep Dive into Inference Economics

Deep analysis of the real cost of serving a 2.8 trillion parameter model. From MoE sparse activation to batching scale effects and inference optimization, revealing why model size and serving cost are less correlated than assumed.

Aug 1, 2026·2 min

The AI Model Value-for-Money Battle: How to Rationally Choose the Right LLM for You

As AI LLM capabilities converge, cost-effectiveness becomes the key selection factor. This article explores how to rationally compare AI models through value assessment, task matching, and cost-benefit analysis.

Aug 1, 2026·4 min

Heretic Uncensoring Technology Explained: Practical Applications on Jamba2 and Qwen3.5 Open-Source Models

Deep dive into Heretic uncensoring technology applied to Jamba2-Mini, Qwen3.5-9B, and 27B open-source models, exploring how refusal rates dropped from 97% to 4% and the safety debates involved.

Jul 31, 2026·3 min

Why We Abandoned Our LLM Router: Lessons in Over-Engineering

An in-depth analysis of why teams are abandoning LLM routers, exploring hidden complexity costs, outdated cost assumptions, and how to avoid over-engineering in AI systems.

Jul 31, 2026·3 min

Orca-Bench: Evaluating Whether AI Agents Can Handle Production Oncall Duties

Orca-Bench is a benchmark for evaluating AI agents' operational capabilities, testing LLMs on fault diagnosis, multi-tool orchestration, and risk decisions in simulated Oncall scenarios.

Jul 31, 2026·3 min

Practical Guide to Setting Up a Local AI Coding Environment on MacBook Pro M4

Complete guide to setting up a local AI coding environment on MacBook Pro M4, covering Ollama, MLX, Continue, Qwen3-Coder 30B configuration, and performance optimization strategies for 32GB RAM.

Jul 31, 2026·5 min

The Complete Learning Path to Understanding the Kimi K3 Technical Report

A systematic learning path for understanding the Kimi K3 technical report, covering MoE, MLA, distributed training, and modern post-training techniques.

Jul 31, 2026·3 min

GPT-5.6 Luna Review: Outperforms Google's Flagship at a Lower Price

GPT-5.6 Luna tops Google's flagship on the Artificial Analysis Intelligence Index while priced below Google's entry-level model. A deep dive into what this performance-cost breakthrough means.

Jul 31, 2026·2 min

DeepSeek V4 Flash Weights Open-Sourced: A New Option for Lightweight LLM Deployment

DeepSeek V4 Flash model weights reportedly open-sourced. This article analyzes its lightweight positioning, open-weight value, comparisons with closed-source models, and deployment guidance.

Jul 31, 2026·2 min

Running Kimi K3 with 29GB of RAM: The Trade-offs and Costs of Extreme Quantization

Running Kimi K3 with 29GB RAM at just 0.5 tok/s. An in-depth analysis of extreme quantization techniques, performance trade-offs, and the impossible triangle of local LLM deployment.

Jul 31, 2026·2 min

Running Kimi K3 in 29GB of RAM: The Trade-offs and Costs of Extreme Quantization

Running Kimi K3 with 29GB RAM at just 0.5 tok/s. A deep analysis of extreme quantization techniques, performance trade-offs, and the impossible triangle of local LLM deployment.