#KV cache optimization
315 related articles
·4 min
KV Cache Saves 20x on Costs: The Underlying Principles and Practical Tips for LLM Inference Optimization
Deep dive into how KV Cache reduces LLM API costs by 20x. From Transformer attention matrix multiplication overhead to prompt caching best practices, understand the fundamentals of AI inference cost optimization.
Read more →
·3 min
LLM Cost Optimization in Practice: Balancing Savings with User Experience
Practical LLM cost optimization strategies covering Prompt trimming, context compression, and multi-model routing to cut Token costs while maintaining output quality at scale.
Read more →
·3 min
LLM Cost Optimization in Practice: Balancing Savings and User Experience
Practical strategies for LLM cost optimization: prompt trimming, context compression, multi-model routing, and more to cut token costs while maintaining output quality at scale.
Read more →
·3 min
Is LLM Inference Really That Expensive? Breaking Down the Cost Reduction Logic of GPU Compute and Inference Frameworks
Analyzing real LLM inference costs: from B200 GPU compute gains, vLLM framework optimization to MTP multi-token prediction, explaining why serving costs are widely overestimated.
Read more →
·4 min
Best Practices Guide for Deploying Nextcloud in a 15-Person Small Business
Complete Nextcloud deployment guide for 15-person small businesses covering AIO installation, hardware specs, Redis and PHP tuning, OnlyOffice vs Collabora, backup strategies, and more.
Read more →
·4 min
Best Practices Guide for Deploying Nextcloud in a 15-Person Small Business
Complete Nextcloud deployment guide for 15-person small businesses covering AIO installation, hardware specs, Redis and PHP tuning, OnlyOffice vs Collabora, backup strategies, and key practices for building a stable private cloud office platform.
Read more →
·2 min
Behind the Open-Source Model Frenzy: Who Will Provide Cheap Inference Services?
Open-source LLM weights don't equal low-cost access for developers. This article analyzes the inference service gap in open-source AI and how providers like Together AI and Groq are addressing it.
Read more →
·2 min
Behind the Open-Source Model Frenzy: Who Will Provide Cheap Inference Services?
Open-source LLM weights don't mean developers can use them cheaply. This article examines the inference service gap in open-source AI and how providers like Together AI and Groq are addressing it.
Read more →
·3 min
Self-Hosting Kimi K3: Is Spending 20% More on Hardware Worth a 20% Improvement in Task Performance?
Analysis of whether spending 20% more on hardware for self-hosting Kimi K3 to gain 20% task performance improvement is worthwhile, covering inference precision, VRAM optimization, and tiered deployment.
Read more →
·2 min
Qwen Scribe: A Deep Dive into the Local Speech Transcription Tool for Apple Silicon
Qwen Scribe is a local speech transcription tool optimized for Apple Silicon, running fully offline with Qwen models. Explore its technical features, privacy benefits, and comparison with Whisper.
Read more →
·2 min
Qwen Scribe: A Deep Dive into the Local Speech Transcription Tool for Apple Silicon
Qwen Scribe is a local speech transcription tool optimized for Apple Silicon, powered by Qwen models for fully offline use. Explore its technical features, privacy benefits, and comparison with Whisper.
Read more →
·3 min
Complete Guide to Running Claude Code with Ollama: API Error Troubleshooting & Local Model Optimization
Guide to running Claude Code via Ollama locally: troubleshooting API errors, output token limits, model freezes, with model selection, parameter tuning, and alternative tool recommendations.
Read more →
·2 min
Real User Feedback: Claude Opus 5 vs Gemini 3.1 Pro — Does Speed Determine the Experience?
A Reddit user's hands-on comparison of Claude Opus 5 vs Gemini 3.1 Pro reveals that response speed and interaction fluidity may matter more than raw intelligence in choosing an LLM.
Read more →
·3 min
Local LLM Agent Lagging? Lowering num_gpu Is the Key
Local LLM crashing in Agent frameworks? The issue may be num_gpu set too high. Learn what num_gpu really controls (GPU layer offloading, not GPU count) and how to tune it for stable Agent performance.
Read more →
·3 min
Production-Grade LLM Inference Deployment: A Practical K8s Handbook
Complete guide to deploying production-grade LLM inference on Kubernetes, covering GPU scheduling, vLLM engine selection, autoscaling, observability, and cost optimization.
Read more →
·3 min
Is LLM Inference Actually Profitable? Breaking Down the Cost Structure and Business Logic of Large Model Inference
A deep dive into LLM inference cost structure and profitability models—from GPU throughput, MoE architecture, and KV Cache to scale effects—revealing the business logic behind API price wars.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.
Read more →
·3 min
Kimi K3 vs Claude Real-World Test: Near-Equal Performance at 1/8 the Price
Real-world comparison of Kimi K3 vs Claude flagship across e-commerce pages, 3D fighting games, and flight simulators. Kimi K3 delivers 90% output quality at 1/8 the price with faster speeds and local deployment support.
Read more →
·3 min
Kimi K3 Launch: A 2.8 Trillion Parameter Flagship Model Makes Its Grand Entrance
Moonshot AI launches Kimi K3 with 2.8 trillion parameters and 1M token context. Google delays Gemini 3.5 Pro, AI coding tools upgrade collectively as competition shifts to coding and Agent capabilities.
Read more →