#VRAM usage
303 related articles
Tutorials·2 min
DeepSeek Local Deployment Tutorial: One-Click Install & Run with Ollama
Learn how to deploy DeepSeek R1 locally using Ollama with this step-by-step guide covering installation, model selection, hardware requirements, and advanced features like GUI and knowledge bases.
Read more →
Tutorials·2 min
How to Tell If Your GPU Is Out of VRAM When Running Local LLMs: A Complete Guide to VRAM Monitoring
How to tell if your GPU is out of VRAM when running local LLMs. Learn the difference between dedicated and shared GPU memory, monitor VRAM overflow via Task Manager, and use quantization and context length control to avoid OOM.
Read more →
Product Reviews·1 min
DeepSeek V4 vs Qwen3.6 Real-World Testing: In-Depth Comparison Across 8 Categories and 150 Scenarios
BenchLocal real-world testing of DeepSeek V4 Pro, V4 Flash vs Qwen3.6 27B across 8 categories and 85 scenarios. V4 Pro leads by 6% but stumbles on math reasoning. Qwen3.6 Q6 rivals V4 Pro in agent tasks.
Read more →
Product Reviews·1 min
Qwen3.6 Quantized Local Deployment Benchmark: Which Is Best — NVFP4, APEX, Q4, or Q6?
Benchmarking 7-8 Qwen3.6 quantized models across 8 dimensions including tool calling, CLI ops, and bug fixing. Comparing NVFP4, APEX, Q4, Q6 with rankings and recommendations.
Read more →
Tutorials·2 min
Hermes + Qwen3.6 Local Deployment Tutorial: Build Your Private AI Agent at Zero Cost
Step-by-step tutorial to deploy Hermes Agent with Qwen3.6 open-source LLM locally. Covers WSL setup, model download, Telegram bot integration for a zero-cost private AI Agent.
Read more →
Product Reviews·1 min
Qwen3.6 27B: Real-World Testing of Three Unconventional Quantized Models — Code Performance +15.8PP, 40B Distillation, 16GB Adaptation
Real-world comparison of three community-built Qwen3.6 27B variants: OmniMerge V4 with +15.8pp code gains, 40B OPUS distilled for roleplay, and a 16GB-optimized version for limited VRAM.
Read more →
Tutorials·2 min
vLLM and SGLang Local Deployment Tutorial: A Practical Guide to 3-8x Performance Gains
Complete guide to deploying vLLM and SGLang locally. Compare performance vs LM Studio, deploy in 3 steps with Docker + AI assistant. Covers SGLang vs vLLM selection, 5090 VRAM optimization, and Cherry Studio integration.
Read more →
Tech Frontiers·1 min
Wukong 2.2P Open Source: 35B MOE Model Outperforms Qwen3.6-27B at 3–5x the Speed
Wukong 2.2P 35B MOE model is now open source. Using adversarial hybrid distillation, it outperforms Qwen3.6-27B. Runs at 158 tokens/s on RTX 4090 with only 8.9GB VRAM. Supports 256K context.
Read more →
Tech Frontiers·1 min
Qwen3.6 MTP Acceleration Benchmarked: Single GPU Inference Hits 220 token/s
Qwen3.6 experimental MTP-GGUF benchmarked: single GPU pushes 35B-A3B model to 220 token/s, 1.4x faster with zero accuracy loss. Covers MTP principles, optimal Draft Tokens strategy, and RTX 5090 results.
Read more →
Industry Insights·2 min
Enterprise LLM Selection Guide: In-Depth Comparison of Llama 3.1, Qwen 2.5, and DeepSeek
How should enterprises choose open-source LLMs? This guide compares Llama 3.1, Qwen 2.5, DeepSeek, and Mistral across model capabilities, hardware requirements, and business scenarios.
Read more →
Deep Dives·2 min
How Fast Is 10 Tokens Per Second from an LLM? An Open-Source Tool for Visualizing Token Generation Speed
An open-source tool lets you visualize LLM token generation speeds from 5-800 TPS, helping developers make rational choices about model selection and inference optimization.
Read more →
Tutorials·2 min
Local Deployment of DeepSeek + RAG Knowledge Base Tutorial: A Step-by-Step Guide with Ollama + RAGFlow
Step-by-step guide to locally deploying DeepSeek with Ollama and building a RAG private knowledge base with RAGFlow. Covers environment setup, Docker deployment, and Embedding principles.
Read more →
Tutorials·2 min
Build Custom ComfyUI Nodes with AI from Scratch: A Complete Hands-On Tutorial
Learn how to create custom ComfyUI nodes with zero coding experience using AI tools like Claude and Gemini. Covers node structure, AI tool selection, and hands-on development.
Read more →
Product Reviews·4 min
SparkWinShape Review: Is Windsurf Auto Account-Switching for Unlimited Claude Opus Actually Reliable?
Real-world test of SparkWinShape plugin for Windsurf auto account-switching to use Claude Opus unlimited. Covers workflow, core features, risk analysis, and compliant alternatives.
Read more →
Product Reviews·3 min
Pixal3D Hands-On Comparison with Tripl3/Trellis/Hunyuan: Full Analysis of Pixel-Perfect Alignment Pros and Cons
In-depth hands-on test of Tencent's open-source Pixal3D 3D generation model, analyzing pixel-level alignment technology with multi-model comparisons against Trellis 2, Hunyuan, and Tripl3.
Read more →
Tutorials·1 min
Guide to Token Freedom: 5 Money-Saving Strategies for Using Codex and GPT at Low Cost
A detailed guide to achieving Token freedom with 5 strategies: model selection, relay APIs, Prompt optimization, free tier usage, and more to slash GPT and Codex API costs.
Read more →
Tutorials·2 min
Anima Anime AI Model: Local Deployment with 6GB VRAM & Workflow Configuration Guide
Complete guide to locally deploying the Anima anime AI model with just 6GB VRAM. Covers ComfyUI workflow setup, txt2img parameters, upscaling tips, and low-VRAM optimization for mid-range GPUs.
Read more →
Deep Dives·4 min
How Dangerous Are Windsurf Account-Switching Plugins? A Deep Dive into Technical Principles and Security Risks
Deep analysis of Windsurf seamless account-switching plugin's technical implementation, revealing risks of account bans, code theft, and legal liability. Four compliant alternatives for developers.
Read more →
Tutorials·4 min
NVIDIA Model Optimizer Post-Training Quantization (PTQ) Practical Guide
A deep dive into NVIDIA Model Optimizer's PTQ workflow, covering INT8/INT4 quantization principles, calibration methods, RTX GPU optimization, and best practices for deploying quantized LLMs on consumer GPUs.
Read more →
Product Reviews·3 min
AnythingLLM: A Deep Dive into the All-in-One Local AI Knowledge Base Tool
Deep dive into AnythingLLM, a privacy-first, zero-config local AI productivity platform. Supports RAG document chat, multi-model integration, knowledge bases, and AI Agents with nearly 60K GitHub stars.
Read more →