#quantized model deployment
307 related articles
Tutorials·1 min
DeepSeek + RAGFlow Local Deployment AI Knowledge Base Tutorial (2025 Latest)
Step-by-step guide to locally deploy a personal AI knowledge base with DeepSeek + RAGFlow + Ollama. Covers RAG principles, Ollama setup, Docker deployment, and knowledge base optimization.
Read more →
Tech Frontiers·4 min
Step 3.5 Flash Deep Dive: How a 196B Parameter Open-Source Model Beat Gemini
Deep dive into StepFun's Step 3.5 Flash: 196B parameter MoE model activating only 11B, 350 tokens/sec coding speed, 256K context window, local deployment ready. How it beats Gemini 3 Flash.
Read more →
Tutorials·4 min
NVIDIA Model Optimizer Post-Training Quantization (PTQ) Practical Guide
A deep dive into NVIDIA Model Optimizer's PTQ workflow, covering INT8/INT4 quantization principles, calibration methods, RTX GPU optimization, and best practices for deploying quantized LLMs on consumer GPUs.
Read more →
Deep Dives·2 min
The MARVIS Project: A Complete Analysis of Embedded AI Agents Empowering Autonomous Space Exploration
Deep dive into how the MARVIS project deploys LLM agents on spacecraft, covering agent architecture, edge hardware token performance benchmarks, expert evaluations, and space AI benchmark planning.
Read more →
Tutorials·3 min
Grammar-Constrained Decoding: A Practical Guide to Generating Reliable Bash Code with Small Models
Deep dive into Grammar-Constrained Decoding (GCD) technology: applying Bash syntax constraints during inference to dramatically improve small language models' code generation correctness and executability for AI Agent edge deployment.
Read more →
Product Reviews·3 min
AnythingLLM: A Deep Dive into the All-in-One Local AI Knowledge Base Tool
Deep dive into AnythingLLM, a privacy-first, zero-config local AI productivity platform. Supports RAG document chat, multi-model integration, knowledge bases, and AI Agents with nearly 60K GitHub stars.
Read more →
Product Reviews·4 min
Hugging Face Transformers: Deep Dive into the 160K-Star Open Source AI Model Framework
Deep dive into Hugging Face Transformers: architecture, multimodal support, ecosystem, and trends. Learn how this 160K-Star project became essential for AI developers.
Read more →
Tutorials·3 min
Connect Personal WeChat to AI: Screenshot + OCR Solution for Auto-Reply in 1 Hour
Low-risk personal WeChat AI integration via screenshot + OCR + hotkey simulation. Includes three approach comparisons, Ollama local Qwen vision model deployment, and solutions for infinite loops and cursor flicker issues.
Read more →
Product Reviews·3 min
Sim Studio: Deep Dive into the Open-Source AI Agent Workflow Platform with Nearly 10K Stars
Deep analysis of open-source AI workflow platform Sim Studio with nearly 10K GitHub Stars. Apache 2.0 licensed, supports full local deployment and Ollama local LLM integration. Compared with Dify and n8n.
Read more →
Tech Frontiers·4 min
Claude's Quota Slashed by 20x — Can GLM 4.6 Be the Best Alternative?
Anthropic slashes Claude Max Opus quotas by 20x, driving developer exodus. GLM 4.6 enters with 1/10th API pricing, 200K context window & open weights. Full comparison inside.
Read more →
Tutorials·3 min
Ollama + OpenCode Local AI Coding: A Complete Zero-Cost Alternative to Cursor
Tutorial: Deploy Qwen3 Coder locally via Ollama with OpenCode for zero-cost AI coding. Covers setup, code generation, auto-debugging, and hardware recommendations.
Read more →
Product Reviews·3 min
GPT-OSS 120B Local Deployment Review: Comprehensive Comparison with O4 Mini on Code Generation and Reasoning
Hands-on review of OpenAI's first open-weight reasoning model GPT-OSS 120B, locally deployed via Ollama, benchmarked against O4 Mini on code generation, reasoning, and UI design.
Read more →
Tutorials·3 min
GPT-OSS Open-Source Model Local Deployment Tutorial: Real-World Performance and VRAM Usage Analysis
Complete guide to deploying OpenAI's open-source GPT-OSS model locally with Ollama. Real-world testing of the 20B version on RTX 4090 covering Chinese comprehension, logical reasoning, and VRAM usage analysis under MoE architecture.
Read more →
Tutorials·3 min
LocalAI Tutorial: Deploy LLMs Locally Without a GPU Using Docker
Complete LocalAI deployment tutorial: run nearly 1,000 open-source LLMs locally without a GPU. One-click Docker setup, OpenAI API compatible, supports chat, image generation, and voice — fully private.
Read more →
Tutorials·4 min
2025 LLM Career Transition Roadmap: A Three-Phase Plan from Foundations → Frameworks → Real-World Projects
A systematic 2025 LLM career transition roadmap covering Python, Transformers, LangChain, LlamaIndex, RAG, Agent development, and fine-tuning across three phases achievable in 2-3 months.
Read more →
Tech Frontiers·3 min
Hugging Face Open-Source Agent Ecosystem Explained: From Local Deployment to AI-Automated Training
Deep dive into Hugging Face's open-source Agent ecosystem: open models matching closed-source performance, local deployment options, Skills for conversational model training, and MCP integration.
Read more →
Tutorials·3 min
DeepSeek R1 + BrowserUse: A Free Open-Source Browser Automation Setup Tutorial
Learn how to build a free browser automation solution with DeepSeek R1 and BrowserUse. Includes Ollama local deployment, WebUI setup, and real-world tests rivaling OpenAI Operator.
Read more →
Tutorials·3 min
n8n One-Click Deployment of 30+ AI Tools: A Free Self-Hosted Alternative to Zapier
Explore the GitHub project n8n-install: deploy n8n, Ollama, Flowise, and 30+ AI tools with one command. A free self-hosted alternative to Zapier and Make.
Read more →
Tutorials·3 min
4-bit QLoRA Fine-tuning LLaMA 3 in Practice: A Guide to Training 8 Billion Parameter LLMs on Consumer GPUs
Learn how to fine-tune LLaMA 3 8B on consumer GPUs using QLoRA and Unsloth. Covers 4-bit quantization, LoRA adaptation, Alpaca data format, training pipeline setup, and model deployment—runs on RTX 3090.
Read more →
Product Reviews·3 min
Cube Studio: A Comprehensive Analysis of Tencent's Open-Source Cloud-Native AI Platform
Deep dive into Tencent Music's open-source Cube Studio cloud-native AI platform, covering distributed training, LLM fine-tuning & inference, Pipeline orchestration, and domestic hardware adaptation.
Read more →