#quantized model inference
365 related articles
·3 min
LLM Development Learning Path: From Zero to Enterprise-Level Projects
A complete LLM development learning roadmap covering prompt engineering, RAG, AI Agents, and fine-tuning — helping beginners master LangChain, LlamaIndex, and more.
Read more →
·6 min
A 284B Model Running on a MacBook: The Real Breakthrough and Barriers of Local DeepSeek Inference
Redis creator runs 284B-parameter DeepSeek model on a MacBook Pro at 26 tokens/sec using a pure C engine, asymmetric quantization, and MoE architecture.
Read more →
·3 min
Running Local LLMs on AMD GPUs: Ditching Ollama for LM Studio Delivers 7x Speed Boost
AMD GPU black screens running local LLMs? This post-mortem covers Ollama's 3 fatal flaws and how switching to LM Studio boosted token speed from 5 to 36, with ROCm setup, Speculative Decoding, and GFX version tips.
Read more →
·2 min
Ollama: The 175K-Star Tool for Running Local LLMs with One Command
Ollama is an open-source local LLM runner with 175K+ GitHub Stars. Built in Go, it supports Llama, Mistral, Qwen and more — deploy in 3 steps, no setup headaches.
Read more →
·2 min
Why Banks and Hospitals Must Use Local AI: A Complete Guide to Private Large Model Deployment
Why do banks and hospitals build Local AI instead of using cloud services? This guide covers the full tech stack — Ollama, RAG, vector databases — and real-world enterprise deployment use cases.
Read more →
·3 min
Needle: Fine-Tuning a 26M Ultralight Tool-Calling Model Locally — Full Walkthrough
Needle is a 26M-parameter tool-calling model. Learn how to replace Gemini with Ollama for local training data generation and fine-tune on a single GPU, achieving 96.7% F1 — ideal for edge AI deployment.
Read more →
·7 min
AI Agent Memory Layer Comparison: Mem0, Zep, Letta, and Cloudflare — Four Major Solutions Evaluated
In-depth comparison of four AI agent memory layer solutions: Mem0's extract-retrieve approach, Zep's temporal knowledge graphs, Letta's self-editing memory, and Cloudflare Durable Objects as infrastructure primitives.
Read more →
·8 min
The Enterprise AI Flywheel: Building a Model-Eval Loop as a Competitive Moat
The core of enterprise AI isn't calling general models—it's building a self-reinforcing "model-harness-sandbox-eval" flywheel. This article analyzes the four components, tacit knowledge moats, and the "token value per watt" efficiency metric.
Read more →
·3 min
Qwythos-9B Hands-On Review: Deploying an Uncensored Open-Source Model That Inherits Claude's Reasoning Abilities
Hands-on review of Qwythos-9B, distilled from 500M+ Claude reasoning traces. Supports 1.04M token context, uncensored, runs on just 4GB VRAM. Full deployment guide included.
Read more →
·4 min
The Complete Guide to Running GLM-5.2 Locally with 4 RTX 3090s for $4,000
How to build a local AI inference server with 4 used RTX 3090 SXM4 GPUs to run GLM-5.2 via Llama.cpp and Unsloth IQ quantization, with real benchmarks on speed and quality.
Read more →
·3 min
Vibe Coding Practical Guide: Tool Selection, Development Workflow, and Lessons Learned
In-depth analysis of Vibe Coding methodology covering Claude Code, Cursor, and Codex tool strategies, full development workflow tips, and four key principles for avoiding pitfalls in complex systems.
Read more →
·2 min
XiaoWu Voice Input Method: Powered by Local LLMs for Accurate Offline Speech Recognition
XiaoWu is a fully local AI voice input method powered by on-device LLMs for accurate offline speech recognition, smart punctuation, and minimalist interaction — no internet required.
Read more →
·3 min
NVIDIA XR AI Platform Explained: Full-Stack AI Agent Development for AR Glasses
Deep dive into how NVIDIA's XR AI platform enables AI Agent development for AR glasses through cloud-edge architecture, covering visual perception, voice interaction, and multimodal reasoning.
Read more →
·2 min
Nex N2 Pro In-Depth Review: How Does This Chinese Open-Source Agent Model Really Perform?
In-depth review of Nex N2 Pro, a Chinese open-source Agent model. Covers frontend code generation, Agent workflows, and benchmark comparisons, revealing gaps between official claims and independent tests.
Read more →
·3 min
OpenLLMVTuber: A Deep Dive into the Open-Source AI Virtual Character Framework
Deep dive into OpenLLMVTuber, a 10K-star open-source AI virtual character framework integrating ASR, LLM, TTS, and Live2D with voice interruption, visual perception, and modular architecture.
Read more →
·3 min
Claude Code Desktop Hands-On: Transparent Context Window & CC Switch for Compute Freedom
Hands-on review of Claude Code Desktop's transparent context window, multi-project heatmap, efficiency panel, and CC Switch open-source routing gateway for model freedom.
Read more →
·4 min
Llama.cpp Windows Local Deployment Guide: Run LLMs in Three Steps Without Compiling
Step-by-step guide to deploying Llama.cpp on Windows without compiling. Download pre-built packages, configure CUDA, and run GGUF quantized models locally with GPU acceleration and web UI in three simple steps.
Read more →
·3 min
Spring AI + Ollama: Run Large Language Models Locally at Zero Cost
Learn how to integrate Spring AI with Ollama to run open-source LLMs like Llama and Gemma locally for free. Covers setup, configuration, and code — switch from OpenAI by just changing dependencies.
Read more →
·3 min
ThundeRobot Hunter Blade S 2026 Review: i9-13900HX + RTX 5060 Configuration Analysis & Buying Guide
In-depth review of the ThundeRobot Hunter Blade S 2026 with i9-13900HX and RTX 5060 16GB. Analyzing CPU/GPU performance, AI capabilities, and value at ~6,671 RMB after subsidies.
Read more →
·3 min
Fireworks AI Launches Qwen 3.7 Plus: Zero Data Retention and 99.9% SLA for Enterprise Deployment
Fireworks AI launches Qwen 3.7 Plus with latency/throughput optimization, zero data retention, and 99.9% SLA enterprise guarantees. Explore the full-stack deployment solution for commercial open-source model inference.
Read more →