#KV Cache
379 related articles
·4 min
From Enter to First Token: Decrypting the Full LLM Inference Pipeline (Autoregression, KV Cache & Decoding Strategies)
From pressing Enter to the first character appearing, what happens inside an LLM? This article breaks down autoregressive generation, KV cache acceleration, and decoding strategies like temperature, Top-k, and Top-p.
Read more →
·4 min
KV Cache Saves 20x on Costs: The Underlying Principles and Practical Tips for LLM Inference Optimization
Deep dive into how KV Cache reduces LLM API costs by 20x. From Transformer attention matrix multiplication overhead to prompt caching best practices, understand the fundamentals of AI inference cost optimization.
Read more →
Tech Frontiers·1 min
Cloudflare Contributes Critical KV Cache and Mooncake Fixes to SGLang
Cloudflare contributes decode KV cache offload and Mooncake recovery fixes to SGLang, resolving garbled output under high concurrency for Kimi K2.6 and enabling automatic fault recovery in distributed inference.
Read more →
·4 min
Best Practices Guide for Deploying Nextcloud in a 15-Person Small Business
Complete Nextcloud deployment guide for 15-person small businesses covering AIO installation, hardware specs, Redis and PHP tuning, OnlyOffice vs Collabora, backup strategies, and more.
Read more →
·4 min
Best Practices Guide for Deploying Nextcloud in a 15-Person Small Business
Complete Nextcloud deployment guide for 15-person small businesses covering AIO installation, hardware specs, Redis and PHP tuning, OnlyOffice vs Collabora, backup strategies, and key practices for building a stable private cloud office platform.
Read more →
·2 min
Behind the Open-Source Model Frenzy: Who Will Provide Cheap Inference Services?
Open-source LLM weights don't equal low-cost access for developers. This article analyzes the inference service gap in open-source AI and how providers like Together AI and Groq are addressing it.
Read more →
·2 min
Behind the Open-Source Model Frenzy: Who Will Provide Cheap Inference Services?
Open-source LLM weights don't mean developers can use them cheaply. This article examines the inference service gap in open-source AI and how providers like Together AI and Groq are addressing it.
Read more →
·3 min
Self-Hosting Kimi K3: Is Spending 20% More on Hardware Worth a 20% Improvement in Task Performance?
Analysis of whether spending 20% more on hardware for self-hosting Kimi K3 to gain 20% task performance improvement is worthwhile, covering inference precision, VRAM optimization, and tiered deployment.
Read more →
·2 min
Qwen Scribe: A Deep Dive into the Local Speech Transcription Tool for Apple Silicon
Qwen Scribe is a local speech transcription tool optimized for Apple Silicon, running fully offline with Qwen models. Explore its technical features, privacy benefits, and comparison with Whisper.
Read more →
·2 min
Qwen Scribe: A Deep Dive into the Local Speech Transcription Tool for Apple Silicon
Qwen Scribe is a local speech transcription tool optimized for Apple Silicon, powered by Qwen models for fully offline use. Explore its technical features, privacy benefits, and comparison with Whisper.
Read more →
·3 min
Complete Guide to Running Claude Code with Ollama: API Error Troubleshooting & Local Model Optimization
Guide to running Claude Code via Ollama locally: troubleshooting API errors, output token limits, model freezes, with model selection, parameter tuning, and alternative tool recommendations.
Read more →
·2 min
Real User Feedback: Claude Opus 5 vs Gemini 3.1 Pro — Does Speed Determine the Experience?
A Reddit user's hands-on comparison of Claude Opus 5 vs Gemini 3.1 Pro reveals that response speed and interaction fluidity may matter more than raw intelligence in choosing an LLM.
Read more →
·3 min
Local LLM Agent Lagging? Lowering num_gpu Is the Key
Local LLM crashing in Agent frameworks? The issue may be num_gpu set too high. Learn what num_gpu really controls (GPU layer offloading, not GPU count) and how to tune it for stable Agent performance.
Read more →
·3 min
Production-Grade LLM Inference Deployment: A Practical K8s Handbook
Complete guide to deploying production-grade LLM inference on Kubernetes, covering GPU scheduling, vLLM engine selection, autoscaling, observability, and cost optimization.
Read more →
·3 min
Is LLM Inference Actually Profitable? Breaking Down the Cost Structure and Business Logic of Large Model Inference
A deep dive into LLM inference cost structure and profitability models—from GPU throughput, MoE architecture, and KV Cache to scale effects—revealing the business logic behind API price wars.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.
Read more →
·3 min
Kimi K3 vs Claude Real-World Test: Near-Equal Performance at 1/8 the Price
Real-world comparison of Kimi K3 vs Claude flagship across e-commerce pages, 3D fighting games, and flight simulators. Kimi K3 delivers 90% output quality at 1/8 the price with faster speeds and local deployment support.
Read more →
·3 min
Kimi K3 Launch: A 2.8 Trillion Parameter Flagship Model Makes Its Grand Entrance
Moonshot AI launches Kimi K3 with 2.8 trillion parameters and 1M token context. Google delays Gemini 3.5 Pro, AI coding tools upgrade collectively as competition shifts to coding and Agent capabilities.
Read more →
·3 min
Kimi K3 Open-Source Release: 2.8 Trillion Parameters Tops Global Programming Leaderboard
Moonshot AI's Kimi K3 launches with 2.8 trillion parameters, tops LMArena frontend coding leaderboard as world #1, completing tasks at one-third competitors' cost. Fully open-source for commercial use.
Read more →