#mixed-precision inference
79 related articles
·4 min
Llama.cpp Windows Local Deployment Guide: Run LLMs in Three Steps Without Compiling
Step-by-step guide to deploying Llama.cpp on Windows without compiling. Download pre-built packages, configure CUDA, and run GGUF quantized models locally with GPU acceleration and web UI in three simple steps.
Read more →
·2 min
A Complete Guide to LLM Infrastructure: Core Challenges from GPU Clusters to Inference Optimization
A deep dive into core challenges and key technologies for LLM infrastructure, covering GPU cluster management, inference optimization, distributed training, cost control, and observability.
Read more →
·3 min
Hands-On Testing of DS4 Engine by Redis Creator: How Does DeepSeek V4 Perform Locally on a 128GB Mac?
Redis creator Antirez's DS4 inference engine tested: running DeepSeek V4 Flash locally on a 128GB Mac via asymmetric structure-aware quantization, with real-world coding benchmarks.
Read more →
·3 min
NVIDIA RTX PRO 4500 Accelerates Genomics and Protein Folding Workloads
Explore how the NVIDIA RTX PRO 4500 Blackwell workstation GPU accelerates genomic analysis and protein folding prediction, and its critical role in precision medicine and drug discovery.
Read more →
Industry Insights·4 min
In-Depth Analysis of the AI Large Model Job Market: Two Core Directions and Future Trends
In-depth analysis of the AI large model job market, breaking down the two core directions—algorithm research and engineering deployment—covering requirements, barriers, and career prospects.
Read more →
Industry Insights·2 min
NVIDIA Blackwell Sets New STAC-AI Records for Financial LLM Inference
NVIDIA Blackwell GPU sets new LLM inference records in STAC-AI financial benchmark. Explore Blackwell architecture advantages, TensorRT-LLM co-optimization, and LLM applications in trading and risk management.
Read more →
Tutorials·4 min
NVIDIA Model Optimizer Post-Training Quantization (PTQ) Practical Guide
A deep dive into NVIDIA Model Optimizer's PTQ workflow, covering INT8/INT4 quantization principles, calibration methods, RTX GPU optimization, and best practices for deploying quantized LLMs on consumer GPUs.
Read more →
Deep Dives·5 min
Hugging Face Transformers: A Comprehensive Guide to the 160K-Star Open-Source AI Framework
Comprehensive guide to Hugging Face Transformers, the 160K-star GitHub framework—covering architecture, multimodal support, quantization, and inference optimization for loading, fine-tuning, and deploying pre-trained models.
Read more →
Product Reviews·4 min
Hugging Face Transformers: Deep Dive into the 160K-Star Open Source AI Model Framework
Deep dive into Hugging Face Transformers: architecture, multimodal support, ecosystem, and trends. Learn how this 160K-Star project became essential for AI developers.
Read more →
Tutorials·3 min
NVIDIA Nemotron Labs Open Source Contribution Guide: A Complete Playbook for Developers Joining AI Open Source Projects
A comprehensive guide to contributing to NVIDIA Nemotron Labs open source projects, covering NeMo framework contributions, community participation, and career benefits for AI developers.
Read more →
Deep Dives·4 min
Hugging Face Transformers: Deep Dive into the 160K-Star Open Source AI Framework
Deep dive into Hugging Face Transformers: the 160K-Star open-source framework covering Pipeline API, Auto Classes, multi-modal models, and the full HF ecosystem for AI inference and training.
Read more →
Research·3 min
SVDQuant: 4-bit Quantization Enables Efficient Diffusion Model Inference on Consumer GPUs
SVDQuant, an ICLR 2025 Spotlight paper, achieves 4-bit diffusion model quantization via low-rank decomposition that absorbs outliers, reducing memory by 75%. Open-source engine Nunchaku (3800+ stars) enables FLUX inference on consumer GPUs like RTX 4060.
Read more →
Tutorials·4 min
Hugging Face Transformers: Deep Dive & Practical Guide to the 160K-Star AI Model Framework
Deep dive into Hugging Face Transformers: core features, multi-framework support, 500K+ pretrained models, full-modality task coverage, and hands-on code examples to build AI apps efficiently.
Read more →
Tutorials·3 min
Hugging Face Transformers: The Secret Behind 160K Stars & A Practical Guide
Deep dive into Hugging Face Transformers: the open-source framework with 160K GitHub Stars. Covers full-modality model support, pipeline API, Hub ecosystem, and community mechanisms driving AI democratization.
Read more →
Tutorials·4 min
A Beginner's Guide to Hugging Face Transformers: Deep Dive into the 160K-Star Open Source AI Framework
Deep dive into the Hugging Face Transformers framework: core features, multimodal support, Pipeline & Trainer APIs, ecosystem integration, and how this 160K-Star library powers modern AI development.
Read more →
Product Reviews·5 min
Unsloth: The Open-Source Tool That Speeds Up Local LLM Fine-Tuning by 5x
Unsloth is an open-source LLM training tool with 63,000+ GitHub Stars. It supports local fine-tuning of Gemma 4, Qwen3, DeepSeek and more, with Web UI, VRAM optimization, and 2-5x training speedup on consumer GPUs.
Read more →
Product Reviews·4 min
Unsloth: The Efficient Tool for Local Open-Source LLM Training — The Secret Behind 60K+ GitHub Stars
In-depth analysis of Unsloth, a 60K+ star open-source LLM training tool supporting Gemma 4, Qwen3, DeepSeek local fine-tuning with LoRA/QLoRA to dramatically reduce VRAM requirements.
Read more →
Product Reviews·3 min
Hugging Face Transformers: A Deep Dive into the AI Open-Source Framework Behind 160K Stars
Deep dive into Hugging Face Transformers: core features, Pipeline API usage, ecosystem synergy, large model support, and future challenges of this 160K-star AI framework.
Read more →
