#parallel decoding

156 related articles

Jul 29, 2026·3 min

Is LLM Inference Actually Profitable? Breaking Down the Cost Structure and Business Logic of Large Model Inference

A deep dive into LLM inference cost structure and profitability models—from GPU throughput, MoE architecture, and KV Cache to scale effects—revealing the business logic behind API price wars.

Jul 29, 2026·2 min

ComfyUI KREA 2 v1.2 Identity Edit Workflow: A Practical Guide to Low-VRAM Identity-Preserving Editing

A detailed guide to ComfyUI KREA 2 Identity Edit v1.2 workflow installation and usage, supporting low-VRAM GPUs for identity-preserving editing including pose changes, style transfer, face swap, and virtual try-on.

Jul 28, 2026·3 min

Kimi K3 Reaches 2.5T Parameters, MiniMax M3 Pro Brewing Simultaneously: China's LLM Arms Race Escalates

Kimi K3 adopts new architecture with 2.5T parameters and 1M token context. MiniMax M3 Pro targets 2.7-3T. OpenAI surpasses 7M users, HY-OCR 1.5 achieves SOTA, Amap launches world model.

Jul 28, 2026·4 min

Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop

Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.

Jul 28, 2026·4 min

Laguna S 2.1 Deep Dive: Hands-On Local Deployment of a 118B Parameter Coding Model

In-depth review of Poolside's Laguna S 2.1 open-source coding model: MoE architecture, RL training, DGX Spark local deployment, and real-world agentic coding tests with 8B active parameters.

Jul 27, 2026·5 min

Multiway Turing Machines: A New Paradigm from Single Computation to Multiple Possibilities

An in-depth analysis of Wolfram's multiway Turing machines, exploring how computation expands from single paths to multiway graph structures, and deep connections to AI search algorithms and quantum computing.

Jul 27, 2026·5 min

Ollama Performance Degrades After Connecting to OpenWebUI? A Complete Troubleshooting Guide

Complete guide to troubleshooting Ollama performance degradation with OpenWebUI, covering num_ctx truncation, sampling parameter differences, GPU resource contention, and system prompt interference.

Jul 26, 2026·3 min

Introduction to How AI Large Models Work: A Must-Read Guide for Test Engineers

A clear explanation of how AI large models work: from concept hierarchy and Transformer mechanics to probabilistic traits, helping test engineers grasp AI testing.

Jul 26, 2026·3 min

Introduction to AI Large Language Models: A Must-Read Guide for Test Engineers

A thorough explanation of the essence of AI large language models: from conceptual hierarchy and Transformer mechanics to probabilistic nature, helping test engineers understand LLM strengths and weaknesses.

Jul 25, 2026·2 min

DSpark: DeepSeek Partners with Peking University to Open-Source Inference Acceleration, Boosting Speed by Up to 85%

DeepSeek partners with Peking University to open-source DSpark, an inference acceleration tech boosting single-user speed by 57%-85% under high concurrency. Learn its three core designs and the DSpec framework.

Jul 25, 2026·2 min

DSpark: DeepSeek and Peking University Open-Source Inference Acceleration Tech with Up to 85% Speed Boost

DeepSeek and Peking University open-source DSpark, an inference acceleration tech boosting single-user generation speed by 57%-85% under high concurrency. Learn its 3 core designs and the DSpec framework.

Jul 25, 2026·2 min

D-Flash: How Diffusion Drafting Delivers 3.5x Faster LLM Inference

D-Flash solves the autoregressive drafter latency bottleneck in speculative decoding via fast diffusion parallel drafting and target feature KV injection. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, beating EAGLE3 and MTP.

Jul 25, 2026·3 min

D-Flash: The Secret Behind 3.5x LLM Inference Speedup with Diffusion Drafting

D-Flash uses fast diffusion parallel drafting and target feature KV injection to solve the autoregressive Drafter latency bottleneck in speculative decoding. 16 tokens in just 6ms, up to 3.5x speedup on HumanEval, surpassing EAGLE3 and MTP.

Jul 25, 2026·3 min

Laguna S 2.1: The Open-Source Coding Model That's 14x Smaller Yet Stronger

Poolside releases Laguna S 2.1, an open-source agentic coding model: 118B total params with only 8B active, scoring 40.4% on DeepSWE — ~4.5x DeepSeek V4 Pro Max. Supports 1M-token context, deployable on a single workstation.

Jul 25, 2026·3 min

DeepSeek Open-Sources DeepSpec: A Full-Stack Speculative Decoding Library with 60–85% Inference Speedup

DeepSeek open-sources DeepSpec, a full-stack speculative decoding library delivering 60–85% inference speedup and up to 661% throughput gains in production. MIT license, 9 checkpoints, 9 benchmarks.

Jul 25, 2026·5 min

Why Are China's Open-Source AI Models Getting Stronger? A Deep Dive Into Three Comeback Strategies

Facing US chip bans and closed-source monopoly, how do China's open-source AI models keep fighting back? A deep dive into three core paths: open-source pricing games, optical interconnect positioning, and on-device scenarios.

Jul 25, 2026·5 min

Why Are China's Open-Source AI Models Getting Stronger? A Deep Dive into Three Comeback Paths

Facing US chip bans and closed-source monopolies, how do China's open-source AI models keep striking back? A deep dive into three core paths: open-source pricing-power games, optical interconnect positioning, and edge-side use cases.

Jul 25, 2026·3 min

How China's Open-Source LLMs Are Closing In on America's Closed-Source Giants

Kimi, Alibaba Qwen, and DeepSeek leverage MoE architecture, home-field data advantages, and the open-source flywheel to close in on OpenAI and Anthropic at ultra-low cost. A deep dive into China's open-source LLM catch-up.

Jul 24, 2026·3 min

The Rise of General AI: Breakthroughs in Mathematical Reasoning and Competitive Programming

Frontier AI is going general: costs are dropping, general models are beating specialized ones in math and competitive programming, and multi-agent workflows are maturing fast.

Jul 24, 2026·8 min

The Complete Guide to Agent Testing: Five Core Dimensions and Automation in Practice

An in-depth analysis of the five core dimensions of AI Agent testing: command safety, tool-calling accuracy, task planning, output consistency, and error self-repair. Master automated testing methods and the transition path for test engineers.