#mixed-precision inference

68 related articles

Aug 1, 2026·3 min

DeepSeek V4 Flash Released: Performance Approaching Claude at Just $0.18 per Million Tokens

DeepSeek V4 Flash launches with benchmark scores approaching Claude Opus 4.8 at just $0.18 per million output tokens. Deep analysis of performance, pricing, and industry impact.

Aug 1, 2026·3 min

Reasoning Capabilities Unmatched, Pricing Still Too High: The Dilemma of AI Model Competition

Deep analysis of the dilemma in AI model competition where reasoning gaps and pricing imbalances force vendors to excel at either capability or cost-effectiveness to survive.

Jul 30, 2026·3 min

Reproducing GPT-2: How a Hidden Bug Affected Weight Training

An in-depth analysis of a hidden bug discovered while reproducing GPT-2 from scratch, revealing how implementation errors silently degrade weight quality and sharing practical debugging methodologies.

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Why memory bandwidth (GB/s), not VRAM size, determines local LLM inference speed. Includes tokens/sec formula, GPU bandwidth comparison, and a practical card selection framework.

Jul 30, 2026·3 min

LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec

Learn why memory bandwidth (GB/s)—not VRAM size—determines local LLM inference speed. Get the tokens/sec formula, GPU bandwidth comparisons, and a practical card selection hierarchy.

Jul 30, 2026·2 min

PlayingFild: A Smart Tab Management Chrome Extension Based on Page Content

PlayingFild is a Chrome extension that uses on-device ML to intelligently classify tabs by page content, separating work from leisure with focus timers, idle tab cleanup, and productivity analytics.

Jul 28, 2026·3 min

AMD CDNA5 Architecture Deep Dive: Technical Evolution and the AI Computing Competition Landscape

Deep analysis of AMD's CDNA5 architecture covering Chiplet packaging upgrades, HBM memory evolution, and low-precision compute optimization, examining how AMD challenges NVIDIA's AI chip dominance.

Jul 27, 2026·3 min

DeepSeek-OCR Deployment & Fine-Tuning in Practice: From vLLM Inference to Unsloth Fine-Tuning

Complete guide to DeepSeek-OCR from vLLM inference deployment and Unsloth model loading to fine-tuning, covering cloud server setup, GPU selection, and code examples — all on a single 4090 GPU.

Jul 27, 2026·3 min

Ideogram 4.0 Turbo Workflow: An AI Image Generation Solution Balancing Speed and Quality

Detailed look at the Ideogram 4.0 mixed turbo workflow: RTX 4090 inference in just 15 seconds, rivaling Krea2 speed, with stable output up to 8K resolution.

Jul 27, 2026·3 min

Ideogram 4.0 Turbo Workflow: An AI Image Generation Solution Balancing Speed and Quality

Detailed look at the Ideogram 4.0 mixed turbo workflow: RTX 4090 tested at just 15s inference, matching Krea2 speed with up to 8K resolution output.

Jul 27, 2026·5 min

Ollama Performance Degrades After Connecting to OpenWebUI? A Complete Troubleshooting Guide

Complete guide to troubleshooting Ollama performance degradation with OpenWebUI, covering num_ctx truncation, sampling parameter differences, GPU resource contention, and system prompt interference.

Jul 23, 2026·4 min

The Rise of Chinese Open-Source AI Models: A Head-On Challenge to Closed-Source Dominance

Chinese open-source models rapidly close the capability gap with top closed-source AI. DeepSeek shocks the industry while Qwen matches global benchmarks.

Jul 22, 2026·3 min

V100 GPU Showdown: 4×16G PCIe vs. 2×32G SXM — Which Gives You More Bang for Your Buck?

Benchmarking 4×V100 16G PCIe vs. 2×V100 32G SXM adapter for local LLM inference. Prefill speed, decode speed, power limits, and bandwidth bottlenecks analyzed.

Jul 22, 2026·3 min

Qwen's Six-Generation Evolution: A Complete Breakdown from Architecture Innovations to the Reasoning Race

From Qwen1 to Qwen3-2507: a complete breakdown of six generations of evolution over two years — GQA, MoE, GRPO, dynamic thinking, and the core shift from architecture to reasoning.

Jul 20, 2026·3 min

Running a 27B Model on iPhone: How 1-bit Quantization Achieves Extreme Compression

PrismML's Bonsai compresses a 27B model from 54GB to 3.9GB, running at ~11 tokens/sec on iPhone. A deep dive into QAT, knowledge distillation, and speculative decoding.

Jul 17, 2026·3 min

Open-Source LLM Surge: K3 and V4 Drop Together as the Marginal Cost of Intelligence Approaches Zero

Kimi K3, DeepSeek V4, Liquid, and Mistral are all dropping at once. MXFP4 quantization and MoE architecture are pushing the marginal cost of intelligence toward zero. Here's what it means.

Bonsai 27B: The First 1-bit LLM That R…

Jul 17, 2026·3 min

Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI

Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.

Jul 13, 2026·2 min

Distributed AI Systems: A Practical Engineering Guide to Training, Inference, and Production Deployment

A deep dive into distributed AI systems engineering: data/model/tensor parallelism for training, KV cache, quantization, elastic scaling for inference, and cloud deployment with Kubernetes, Ray, and DeepSpeed.

Jul 12, 2026·2 min

LingBot-Video 1.3B Selective FP8 Quantization Benchmark: 22% Faster Sampling

Reddit developer ALX-CODE shares a selective FP8 quantization scheme for LingBot-Video 1.3B, achieving ~22% faster sampling (4.65s→3.65s) on an RTX 5080. This article breaks down the mixed-precision strategy, open-source resources, and ComfyUI adaptation.

Jul 12, 2026·4 min

Unsloth Releases Qwen3.6 NVFP4 Quantization: 2.5x Faster with Zero Accuracy Loss

Unsloth releases NVFP4 quantization for Qwen3.6 using W4A4 true 4-bit Tensor Core computation, delivering up to 2.5x inference speedup over NVIDIA's official implementation with accuracy matching or exceeding BF16 on benchmarks like MMLU-Pro.