#quantization GGUF
295 related articles
·6 min
A 284B Model Running on a MacBook: The Real Breakthrough and Barriers of Local DeepSeek Inference
Redis creator runs 284B-parameter DeepSeek model on a MacBook Pro at 26 tokens/sec using a pure C engine, asymmetric quantization, and MoE architecture.
Read more →
·3 min
DeepSeek vs Claude Hands-On Comparison: Is the 80x API Price Gap Actually Worth It?
Hands-on comparison of DeepSeek vs Claude across copywriting and business analysis, analyzing the 80x API price gap, data privacy concerns, and self-hosting options.
Read more →
·3 min
Qwythos-9B Hands-On Review: Deploying an Uncensored Open-Source Model That Inherits Claude's Reasoning Abilities
Hands-on review of Qwythos-9B, distilled from 500M+ Claude reasoning traces. Supports 1.04M token context, uncensored, runs on just 4GB VRAM. Full deployment guide included.
Read more →
·4 min
The Complete Guide to Running GLM-5.2 Locally with 4 RTX 3090s for $4,000
How to build a local AI inference server with 4 used RTX 3090 SXM4 GPUs to run GLM-5.2 via Llama.cpp and Unsloth IQ quantization, with real benchmarks on speed and quality.
Read more →
·3 min
Deploy a Free AI Agent Locally: Build an All-in-One Assistant with Qwen 3.5 + Hormis
Learn how to deploy a free AI agent locally using Ollama + Qwen 3.5-9B + Hormis framework, enabling web scraping, desktop control, and file management on consumer hardware.
Read more →
·3 min
AI Large Language Model Learning Roadmap from Scratch: From Beginner to Enterprise-Level Practice
A systematic AI LLM learning roadmap from scratch, covering Python basics, Prompt Engineering, RAG, Agent development, and enterprise-level projects.
Read more →
·3 min
Building AI Agents from Scratch: LangGraph Architecture and Practical Guide
A deep dive into AI agent principles and development practices, covering agent definitions, leading products (Deep Research, ChengPian, Manus), and the complete LangGraph + LangChain + MCP architecture.
Read more →
·2 min
XiaoWu Voice Input Method: Powered by Local LLMs for Accurate Offline Speech Recognition
XiaoWu is a fully local AI voice input method powered by on-device LLMs for accurate offline speech recognition, smart punctuation, and minimalist interaction — no internet required.
Read more →
·3 min
748-Episode AI Large Language Model Development Tutorial: Seven Core Modules from Zero to Project Deployment
Analysis of a 748-episode, 198-hour AI LLM development tutorial covering API integration, prompt engineering, RAG, AI Agents, fine-tuning, multimodal development, and deployment.
Read more →
·4 min
Llama.cpp Windows Local Deployment Guide: Run LLMs in Three Steps Without Compiling
Step-by-step guide to deploying Llama.cpp on Windows without compiling. Download pre-built packages, configure CUDA, and run GGUF quantized models locally with GPU acceleration and web UI in three simple steps.
Read more →
·3 min
Spring AI + Ollama: Run Large Language Models Locally at Zero Cost
Learn how to integrate Spring AI with Ollama to run open-source LLMs like Llama and Gemma locally for free. Covers setup, configuration, and code — switch from OpenAI by just changing dependencies.
Read more →
·3 min
ThundeRobot Hunter Blade S 2026 Review: i9-13900HX + RTX 5060 Configuration Analysis & Buying Guide
In-depth review of the ThundeRobot Hunter Blade S 2026 with i9-13900HX and RTX 5060 16GB. Analyzing CPU/GPU performance, AI capabilities, and value at ~6,671 RMB after subsidies.
Read more →
·4 min
Local Deployment of Claude Code: Principles and Practical Guide for Three Approaches
A detailed guide to locally deploying Claude Code with three approaches (LM Studio, Ollama, vLLM), covering architecture, protocol translation, hardware selection, and model recommendations.
Read more →
·3 min
Deploying a Multimodal AI Agent Locally on a 3080Ti: VRAM Management and a Deep Dive into All Five Modules
A detailed guide to deploying a multimodal AI Agent on a 3080Ti with 12GB VRAM, covering LLM, STT, TTS, image and video generation module selection, dynamic VRAM loading, and real-world performance.
Read more →
·3 min
5 Ways to Deploy LLMs Locally: From Getting Started to Production
A complete guide to 5 local LLM deployment methods: LlamaCPP, Ollama, LM Studio, vLLM/SGLang, and MLX-LM — from personal dev to production environments.
Read more →
·1 min
Gemma 4 12B: Google's Open-Weight Model Runs Locally on Your Laptop
Google releases Gemma 4 12B, an open-weight model that runs locally on laptops. Learn about its performance, local deployment value, and the open-source LLM competitive landscape.
Read more →
·3 min
Gemma 4 Open-Source Model Local Deployment Guide: Ollama Installation & Mobile Setup
Step-by-step guide to deploying Google's Gemma 4 open-source model locally with Ollama and running the lightweight version on mobile with tool calling support.
Read more →
·3 min
Claude Code with Local LLMs: Token-Free Deployment Guide & Configuration
Learn how to connect Claude Code to local LLMs for token-free AI coding. Covers three-layer architecture, Ollama/LM Studio/vLLM setup, protocol translation, and hardware selection.
Read more →
·4 min
AI Engineer Job Search: The Capability Leap from Demo to Production System
AI job demand is surging but companies can't find qualified candidates. Learn the 3 core skills—advanced RAG, local model deployment, and full-stack monitoring—to leap from demo builder to production engineer.
Read more →
·3 min
Claude Code Local Deployment: A Practical Guide to Connecting Local LLMs
Complete guide to deploying Claude Code locally with Ollama, LM Studio, or vLLM. Covers architecture, protocol translation, hardware requirements, and model selection for zero-cost, private AI coding.
Read more →