#VRAM usage

251 related articles

Jul 30, 2026·2 min

Behind the Open-Source Model Frenzy: Who Will Provide Cheap Inference Services?

Open-source LLM weights don't equal low-cost access for developers. This article analyzes the inference service gap in open-source AI and how providers like Together AI and Groq are addressing it.

Jul 29, 2026·3 min

Self-Hosting Kimi K3: Is Spending 20% More on Hardware Worth a 20% Improvement in Task Performance?

Analysis of whether spending 20% more on hardware for self-hosting Kimi K3 to gain 20% task performance improvement is worthwhile, covering inference precision, VRAM optimization, and tiered deployment.

Jul 29, 2026·3 min

Complete Guide to Running Claude Code with Ollama: API Error Troubleshooting & Local Model Optimization

Guide to running Claude Code via Ollama locally: troubleshooting API errors, output token limits, model freezes, with model selection, parameter tuning, and alternative tool recommendations.

Jul 29, 2026·3 min

Local LLM Agent Lagging? Lowering num_gpu Is the Key

Local LLM crashing in Agent frameworks? The issue may be num_gpu set too high. Learn what num_gpu really controls (GPU layer offloading, not GPU count) and how to tune it for stable Agent performance.

Jul 29, 2026·3 min

Echologue: A Privacy-First AI Voice Journal and a Model for Indie Developers

Echologue is a privacy-first AI voice journal that processes data locally with end-to-end encryption. This analysis examines its product design, technical architecture, and indie developer philosophy.

Jul 29, 2026·2 min

ComfyUI KREA 2 v1.2 Identity Edit Workflow: A Practical Guide to Low-VRAM Identity-Preserving Editing

A detailed guide to ComfyUI KREA 2 Identity Edit v1.2 workflow installation and usage, supporting low-VRAM GPUs for identity-preserving editing including pose changes, style transfer, face swap, and virtual try-on.

Jul 28, 2026·3 min

ncnn Vulkan Cross-Platform Edge Inference in Practice: Solving the GPU Fragmentation Deployment Challenge

Facing GPU fragmentation on edge devices, the PostSlate team used ncnn's Vulkan backend for cross-platform ML inference, achieving 10× speedup on RTX 4070 with half the model size and zero runtime installation.

Jul 28, 2026·2 min

HuggingFace Open-Source Voice Assistant: A Complete Guide to Local Deployment

A deep dive into HuggingFace's speech-to-speech open-source project, covering its modular VAD, STT, LLM, and TTS pipeline architecture and the advantages of local deployment for privacy, cost, and latency.

Jul 28, 2026·3 min

Kimi K3 Hands-On Review: 2.5 Trillion Parameter Open-Source Model Challenges Claude

Kimi K3 hands-on review: Moonshot AI's 2.5T parameter MoE model matches Claude in coding, surpasses it in 3D game development, with API pricing at one-tenth the cost of competitors.

Jul 28, 2026·3 min

Kimi K3 Hands-On Review: 2.5 Trillion Parameter Open-Source Model Takes on Claude

Hands-on review of Kimi K3, Moonshot AI's latest 2.5T parameter MoE model. Coding ability ties with Claude, surpasses it in 3D game dev, with API pricing at one-tenth of competitors.

Jul 28, 2026·4 min

Spring AI 2.0 in Practice: Building an Enterprise RAG Knowledge Base Q&A System with Cursor

Build an enterprise RAG knowledge base Q&A system using Spring AI 2.0, Cursor AI programming, Ollama local deployment, and Redis vector storage. Runs on just 4GB VRAM.

Jul 28, 2026·4 min

Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop

Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.

Jul 28, 2026·4 min

Laguna S 2.1 Deep Dive: Hands-On Local Deployment of a 118B Parameter Coding Model

In-depth review of Poolside's Laguna S 2.1 open-source coding model: MoE architecture, RL training, DGX Spark local deployment, and real-world agentic coding tests with 8B active parameters.

Jul 28, 2026·4 min

Kimi K2/K3 Technical Speculation: Deep Dive into RL Hyperparameters and Multi-Teacher Distillation

Deep analysis of RL hyperparameter tuning challenges and 9-policy multi-teacher distillation in Kimi K2/K3 training, exploring the shift from scale to training craft.

Jul 27, 2026·3 min

From RAG to Agent Tuning: A Complete Guide to Deploying LLM Agents

A deep dive into LLM Agent frameworks covering RAG, Agent core components (tools, memory, planning), and Agent Tuning workflows with cost considerations for production deployment.

Jul 27, 2026·3 min

Kimi K3 Launch: Performance Rivaling Claude at One-Third the Price, US-China AI Gap Narrows to 3 Months

Moonshot AI launches Kimi K3 reasoning model with performance rivaling Claude and OpenAI's top models at one-third the price. The US-China AI gap narrows from 6-12 months to just 3 months.

Jul 27, 2026·3 min

Minecraft Java Edition Raises Recommended RAM to 16GB: What the Vulkan Migration Means

Minecraft Java Edition's recommended RAM jumps to 16GB as the game prepares for a Vulkan graphics API migration. Learn what this means for players and the mod ecosystem.

Jul 27, 2026·3 min

Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide

Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.

Jul 27, 2026·3 min

Ollama's Extra Charges for Kimi 3 Spark Community Controversy: Should Open-Source Models Be Free to Use?

Ollama lists Kimi 3 with extra pay-per-use fees, breaking subscription expectations and sparking debate over open-weight models and AI service pricing tiers.

Jul 27, 2026·3 min

Is Ollama Pro Worth It? An In-Depth Comparison of the $20 Subscription vs. API Costs

In-depth analysis of Ollama Pro's $20/month subscription value, comparing usage quotas, equivalent API costs, and ZDR privacy policy to help developers decide if it's worth it.