#mixed-precision inference

79 related articles

Jul 27, 2026·3 min

Ideogram 4.0 Turbo Workflow: An AI Image Generation Solution Balancing Speed and Quality

Detailed look at the Ideogram 4.0 mixed turbo workflow: RTX 4090 inference in just 15 seconds, rivaling Krea2 speed, with stable output up to 8K resolution.

Jul 27, 2026·5 min

Ollama Performance Degrades After Connecting to OpenWebUI? A Complete Troubleshooting Guide

Complete guide to troubleshooting Ollama performance degradation with OpenWebUI, covering num_ctx truncation, sampling parameter differences, GPU resource contention, and system prompt interference.

Jul 23, 2026·4 min

The Rise of Chinese Open-Source AI Models: A Head-On Challenge to Closed-Source Dominance

Chinese open-source models rapidly close the capability gap with top closed-source AI. DeepSeek shocks the industry while Qwen matches global benchmarks.

Jul 22, 2026·3 min

V100 GPU Showdown: 4×16G PCIe vs. 2×32G SXM — Which Gives You More Bang for Your Buck?

Benchmarking 4×V100 16G PCIe vs. 2×V100 32G SXM adapter for local LLM inference. Prefill speed, decode speed, power limits, and bandwidth bottlenecks analyzed.

Jul 22, 2026·3 min

Qwen's Six-Generation Evolution: A Complete Breakdown from Architecture Innovations to the Reasoning Race

From Qwen1 to Qwen3-2507: a complete breakdown of six generations of evolution over two years — GQA, MoE, GRPO, dynamic thinking, and the core shift from architecture to reasoning.

Jul 20, 2026·3 min

Running a 27B Model on iPhone: How 1-bit Quantization Achieves Extreme Compression

PrismML's Bonsai compresses a 27B model from 54GB to 3.9GB, running at ~11 tokens/sec on iPhone. A deep dive into QAT, knowledge distillation, and speculative decoding.

Jul 17, 2026·3 min

Open-Source LLM Surge: K3 and V4 Drop Together as the Marginal Cost of Intelligence Approaches Zero

Kimi K3, DeepSeek V4, Liquid, and Mistral are all dropping at once. MXFP4 quantization and MoE architecture are pushing the marginal cost of intelligence toward zero. Here's what it means.

Bonsai 27B: The First 1-bit LLM That R…

Jul 17, 2026·3 min

Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI

Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.

Jul 13, 2026·2 min

Distributed AI Systems: A Practical Engineering Guide to Training, Inference, and Production Deployment

A deep dive into distributed AI systems engineering: data/model/tensor parallelism for training, KV cache, quantization, elastic scaling for inference, and cloud deployment with Kubernetes, Ray, and DeepSpeed.

Jul 12, 2026·2 min

LingBot-Video 1.3B Selective FP8 Quantization Benchmark: 22% Faster Sampling

Reddit developer ALX-CODE shares a selective FP8 quantization scheme for LingBot-Video 1.3B, achieving ~22% faster sampling (4.65s→3.65s) on an RTX 5080. This article breaks down the mixed-precision strategy, open-source resources, and ComfyUI adaptation.

Jul 12, 2026·4 min

Unsloth Releases Qwen3.6 NVFP4 Quantization: 2.5x Faster with Zero Accuracy Loss

Unsloth releases NVFP4 quantization for Qwen3.6 using W4A4 true 4-bit Tensor Core computation, delivering up to 2.5x inference speedup over NVIDIA's official implementation with accuracy matching or exceeding BF16 on benchmarks like MMLU-Pro.

Jul 12, 2026·5 min

Advanced Machine Learning Reading List: How to Choose the Right Book Among These 4 Classics?

Torn between Géron, Chollet, and Raschka? This article breaks down 4 classic ML books for self-learners aiming at finetuning and small language models (SLM), helping you find the best advanced path.

Jul 12, 2026·3 min

DeepSeek-V4 Flash RL Training Successfully Ported to AMD MI355X GPU: A Complete Breakdown

The Miles team and AMD announce the full port of DeepSeek-V4 Flash RL training to AMD Instinct MI355X GPUs on ROCm, boosting AIME pass@1 from 0.39 to 0.49—a milestone for compute ecosystem diversity.

Jul 12, 2026·6 min

Alibaba Bans Claude Code: The Battle Over AI Coding Tool Security and Data Sovereignty

Alibaba bans Claude Code over security risks, sparking tech community debate. An in-depth analysis of data leakage risks with cloud AI coding tools and the rise of local AI deployment.

Jul 11, 2026·7 min

Practical RL Compute Optimization: Core Strategies to Boost CPU and GPU Utilization

An in-depth look at why CPU and GPU utilization is low in RL training, covering vectorized environment parallelism, distributed Actor-Learner architectures, GPU-side simulation (Isaac Gym/Brax), and Ray RLlib practice.

Jul 11, 2026·3 min

ComfyUI-INT4-Fast: Running Flux Models Smoothly on 6GB GPUs

ComfyUI-INT4-Fast brings W4A4 quantized inference to ComfyUI. RTX 3060 (6GB VRAM) generates 1024×1024 images in 17s. Per-layer mixed-precision routing balances speed and quality for Flux models.

Jul 11, 2026·6 min

GPT-5.6 Now Widely Available: The Era of Three-Tier Models and Multi-Agent Orchestration Arrives

OpenAI releases GPT-5.6 (SOUL/TERRA/LUNA), with Ultra mode running four agents in parallel; Meta launches Muse Spark 1.1 with million-token context; ChatGPT desktop unifies Chat, Work, and Codex.

Jul 11, 2026·5 min

Ollama Raises $65M Series B: 85% of Fortune 500 Have Already Deployed Local LLMs

Local LLM tool Ollama closes a $65M Series B, bringing total funding to $88M. With 9M developers and 85% of Fortune 500 having deployed internally, this deep dive explores why enterprises embrace local LLMs: compliance, Agent cost savings, and open-source ecosystem.

Jul 11, 2026·4 min

Demystifying Unified Memory Architecture: How Mini PCs Run 70B Large Models

Why can a mini PC with unified memory run a 70B model while an RTX 4090 can't? A deep dive into the VRAM wall and unified memory architecture for smarter local AI hardware choices.

Jul 11, 2026·3 min

From IT Support to AI Engineer: A Real NCA-GENL Certification Journey and Lessons Learned

A real NCA-GENL study journal from an IT-support-turned-AI-engineer: 50+ scenario questions, 7-week prep, and a brutal 40% on Trustworthy AI. Covers Transformer concepts, NVIDIA tools, and what actually works.