#FP8 mixed precision
94 related articles
·3 min
DeepSeek V4 Flash Released: Performance Approaching Claude at Just $0.18 per Million Tokens
DeepSeek V4 Flash launches with benchmark scores approaching Claude Opus 4.8 at just $0.18 per million output tokens. Deep analysis of performance, pricing, and industry impact.
Read more →
·3 min
Reasoning Capabilities Unmatched, Pricing Still Too High: The Dilemma of AI Model Competition
Deep analysis of the dilemma in AI model competition where reasoning gaps and pricing imbalances force vendors to excel at either capability or cost-effectiveness to survive.
Read more →
·3 min
GPT-5.6 Luna Review: Outperforms Google's Flagship at a Lower Price
GPT-5.6 Luna tops Google's flagship on the Artificial Analysis Intelligence Index while priced below Google's entry-level model. A deep dive into what this performance-cost breakthrough means.
Read more →
·3 min
Reproducing GPT-2: How a Hidden Bug Affected Weight Training
An in-depth analysis of a hidden bug discovered while reproducing GPT-2 from scratch, revealing how implementation errors silently degrade weight quality and sharing practical debugging methodologies.
Read more →
·3 min
LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec
Learn why memory bandwidth (GB/s)—not VRAM size—determines local LLM inference speed. Get the tokens/sec formula, GPU bandwidth comparisons, and a practical card selection hierarchy.
Read more →
·3 min
LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec
Why memory bandwidth (GB/s), not VRAM size, determines local LLM inference speed. Includes tokens/sec formula, GPU bandwidth comparison, and a practical card selection framework.
Read more →
·3 min
AMD CDNA5 Architecture Deep Dive: Technical Evolution and the AI Computing Competition Landscape
Deep analysis of AMD's CDNA5 architecture covering Chiplet packaging upgrades, HBM memory evolution, and low-precision compute optimization, examining how AMD challenges NVIDIA's AI chip dominance.
Read more →
·3 min
DeepSeek-OCR Deployment & Fine-Tuning in Practice: From vLLM Inference to Unsloth Fine-Tuning
Complete guide to DeepSeek-OCR from vLLM inference deployment and Unsloth model loading to fine-tuning, covering cloud server setup, GPU selection, and code examples — all on a single 4090 GPU.
Read more →
·3 min
HuggingHack Update: S3 Storage, Dual-Engine Scheduling, and GGUF Inspection Get Major Upgrades
HuggingHack releases major updates with S3/MinIO storage, Ollama + vLLM dual-engine scheduling, GGUF inspection, and local accounts for enterprise-grade local LLM management.
Read more →
·3 min
Ideogram 4.0 Turbo Workflow: An AI Image Generation Solution Balancing Speed and Quality
Detailed look at the Ideogram 4.0 mixed turbo workflow: RTX 4090 inference in just 15 seconds, rivaling Krea2 speed, with stable output up to 8K resolution.
Read more →
·3 min
Ideogram 4.0 Turbo Workflow: An AI Image Generation Solution Balancing Speed and Quality
Detailed look at the Ideogram 4.0 mixed turbo workflow: RTX 4090 tested at just 15s inference, matching Krea2 speed with up to 8K resolution output.
Read more →
·5 min
Ollama Performance Degrades After Connecting to OpenWebUI? A Complete Troubleshooting Guide
Complete guide to troubleshooting Ollama performance degradation with OpenWebUI, covering num_ctx truncation, sampling parameter differences, GPU resource contention, and system prompt interference.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics on Model Quantization and Inference Deployment
A systematic guide to must-know AI application engineer interview topics: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment
A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →
·4 min
Qwen-Image 3.0 Released: Image Generation Moves Toward Production-Ready Design
Qwen-Image 3.0 supports 4.5K token instructions, 10px text rendering, and 12-language typography for production-ready posters and infographics. Plus: Anthropic settlement, Grok in Excel, Tencent HRAP 1.0.
Read more →
·4 min
The Rise of Chinese Open-Source AI Models: A Head-On Challenge to Closed-Source Dominance
Chinese open-source models rapidly close the capability gap with top closed-source AI. DeepSeek shocks the industry while Qwen matches global benchmarks.
Read more →
·3 min
V100 GPU Showdown: 4×16G PCIe vs. 2×32G SXM — Which Gives You More Bang for Your Buck?
Benchmarking 4×V100 16G PCIe vs. 2×V100 32G SXM adapter for local LLM inference. Prefill speed, decode speed, power limits, and bandwidth bottlenecks analyzed.
Read more →
·3 min
Running a 27B Model on iPhone: How 1-bit Quantization Achieves Extreme Compression
PrismML's Bonsai compresses a 27B model from 54GB to 3.9GB, running at ~11 tokens/sec on iPhone. A deep dive into QAT, knowledge distillation, and speculative decoding.
Read more →
·3 min
Open-Source LLM Surge: K3 and V4 Drop Together as the Marginal Cost of Intelligence Approaches Zero
Kimi K3, DeepSeek V4, Liquid, and Mistral are all dropping at once. MXFP4 quantization and MoE architecture are pushing the marginal cost of intelligence toward zero. Here's what it means.
Read more →
Bonsai 27B: The First 1-bit LLM That R…
·3 min
Bonsai 27B: The First 1-bit LLM That Runs on Your Phone — A New Milestone for On-Device AI
Bonsai 27B is the first 27B-parameter LLM that runs on smartphones via 1-bit quantization, compressing to 3–4GB. We break down the tech, privacy benefits, and community debate.
Read more →