#CUDA inference setup
65 related articles
·5 min
Building RAG from Scratch: Creating a Domain Expert System with Local Small Models
Step-by-step guide to building a complete RAG pipeline with Ollama + LangChain + FAISS + Qwen 1.5B. Run document retrieval and intelligent Q&A locally without a GPU.
Read more →
·2 min
Getting Started with Ollama: The Go-To Tool for Free Local LLM Deployment
Ollama is a free, open-source LLM management tool supporting macOS, Windows, Linux, and Docker. Deploy DeepSeek and other open-source models locally — no API fees, full data privacy.
Read more →
·6 min
A 284B Model Running on a MacBook: The Real Breakthrough and Barriers of Local DeepSeek Inference
Redis creator runs 284B-parameter DeepSeek model on a MacBook Pro at 26 tokens/sec using a pure C engine, asymmetric quantization, and MoE architecture.
Read more →
·3 min
Running Local LLMs on AMD GPUs: Ditching Ollama for LM Studio Delivers 7x Speed Boost
AMD GPU black screens running local LLMs? This post-mortem covers Ollama's 3 fatal flaws and how switching to LM Studio boosted token speed from 5 to 36, with ROCm setup, Speculative Decoding, and GFX version tips.
Read more →
·2 min
Ollama: The 175K-Star Tool for Running Local LLMs with One Command
Ollama is an open-source local LLM runner with 175K+ GitHub Stars. Built in Go, it supports Llama, Mistral, Qwen and more — deploy in 3 steps, no setup headaches.
Read more →
·5 min
The Complete Guide to OpenCode: Installation, Configuration, and Advanced Usage
A comprehensive guide to OpenCode, the open-source AI coding tool — covering installation (desktop/WSL), model config, rules files, custom commands, MCP extensions, and Agent SQL capabilities.
Read more →
·4 min
The Complete Guide to Running GLM-5.2 Locally with 4 RTX 3090s for $4,000
How to build a local AI inference server with 4 used RTX 3090 SXM4 GPUs to run GLM-5.2 via Llama.cpp and Unsloth IQ quantization, with real benchmarks on speed and quality.
Read more →
·3 min
Deploy a Free AI Agent Locally: Build an All-in-One Assistant with Qwen 3.5 + Hormis
Learn how to deploy a free AI agent locally using Ollama + Qwen 3.5-9B + Hormis framework, enabling web scraping, desktop control, and file management on consumer hardware.
Read more →
·3 min
The Ultimate Ollama Guide: A Complete Walkthrough for Running AI Models Locally
Complete Ollama guide: install and run open-source LLMs like DeepSeek, Llama, and Qwen locally on Windows/Mac/Linux. Free, private, and beginner-friendly.
Read more →
·4 min
Llama.cpp Windows Local Deployment Guide: Run LLMs in Three Steps Without Compiling
Step-by-step guide to deploying Llama.cpp on Windows without compiling. Download pre-built packages, configure CUDA, and run GGUF quantized models locally with GPU acceleration and web UI in three simple steps.
Read more →
·3 min
5 Ways to Deploy LLMs Locally: From Getting Started to Production
A complete guide to 5 local LLM deployment methods: LlamaCPP, Ollama, LM Studio, vLLM/SGLang, and MLX-LM — from personal dev to production environments.
Read more →
·3 min
Gemma 4 Open-Source Model Local Deployment Guide: Ollama Installation & Mobile Setup
Step-by-step guide to deploying Google's Gemma 4 open-source model locally with Ollama and running the lightweight version on mobile with tool calling support.
Read more →
Product Reviews·3 min
Google Gemma 4 Hands-On Review: Offline on Smartphones + Ollama Deployment Tutorial
Hands-on testing of Google Gemma 4 open-source models running offline on three phones, with Dense vs MOE architecture explained and a complete Ollama + Claude Code deployment tutorial.
Read more →
Tutorials·3 min
Why Learning AI from Scratch Leaves You More Confused — A Clear, Systematic Roadmap for Beginners
Confused learning AI from scratch? This guide breaks down why fragmented learning fails and provides a complete path from Python to deep learning with practical tips.
Read more →
Tutorials·4 min
Practical Guide to Building a Local AI Knowledge Base with Qwen3.5 + RAGFlow + Ollama
Step-by-step guide to building a local RAG knowledge base using RAGFlow, Ollama, and LM Studio with Docker, covering Embedding model deployment and network troubleshooting for private AI Q&A.
Read more →
Tutorials·1 min
AI Agent Learning Roadmap: A Complete Guide from LLM Fundamentals to Enterprise-Level Project Implementation
A systematic AI Agent learning roadmap covering Python setup, Prompt Engineering, RAG, LangChain, multi-Agent collaboration, with enterprise medical consultation system case study and phased learning plan.
Read more →
Product Reviews·3 min
Deep Dive into Tencent Marvis: How a System-Level AI Assistant Redefines Human-Computer Interaction
Deep dive into Tencent Marvis system-level AI assistant, analyzing its local knowledge base, semantic search, privacy mode, and how Agents evolve from tools to OS integration.
Read more →
Tutorials·3 min
Running AI Models on a P106 Mining GPU: Build a Local AI Workstation for Under $10
Build a local AI workstation with a P106 mining GPU for under $10. Run Live Portrait and other AI models locally with full privacy, zero marginal cost, and incredible value.
Read more →
Tutorials·3 min
LLM Learning Roadmap: A Complete Guide from Beginner to Project Implementation Across Seven Core Modules
A systematic breakdown of seven core LLM learning modules covering environment setup, Prompt Engineering, RAG, Agents, dev frameworks, fine-tuning, and hands-on projects for developers.
Read more →
Tutorials·2 min
Hermes + Qwen3.6 Local Deployment Tutorial: Build Your Private AI Agent at Zero Cost
Step-by-step tutorial to deploy Hermes Agent with Qwen3.6 open-source LLM locally. Covers WSL setup, model download, Telegram bot integration for a zero-cost private AI Agent.
Read more →