#KV cache optimization
412 related articles
·2 min
Qwen Scribe: A Deep Dive into the Local Speech Transcription Tool for Apple Silicon
Qwen Scribe is a local speech transcription tool optimized for Apple Silicon, powered by Qwen models for fully offline use. Explore its technical features, privacy benefits, and comparison with Whisper.
Read more →
·3 min
Complete Guide to Running Claude Code with Ollama: API Error Troubleshooting & Local Model Optimization
Guide to running Claude Code via Ollama locally: troubleshooting API errors, output token limits, model freezes, with model selection, parameter tuning, and alternative tool recommendations.
Read more →
·2 min
Real User Feedback: Claude Opus 5 vs Gemini 3.1 Pro — Does Speed Determine the Experience?
A Reddit user's hands-on comparison of Claude Opus 5 vs Gemini 3.1 Pro reveals that response speed and interaction fluidity may matter more than raw intelligence in choosing an LLM.
Read more →
·3 min
Local LLM Agent Lagging? Lowering num_gpu Is the Key
Local LLM crashing in Agent frameworks? The issue may be num_gpu set too high. Learn what num_gpu really controls (GPU layer offloading, not GPU count) and how to tune it for stable Agent performance.
Read more →
·3 min
Production-Grade LLM Inference Deployment: A Practical K8s Handbook
Complete guide to deploying production-grade LLM inference on Kubernetes, covering GPU scheduling, vLLM engine selection, autoscaling, observability, and cost optimization.
Read more →
·3 min
Is LLM Inference Actually Profitable? Breaking Down the Cost Structure and Business Logic of Large Model Inference
A deep dive into LLM inference cost structure and profitability models—from GPU throughput, MoE architecture, and KV Cache to scale effects—revealing the business logic behind API price wars.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.
Read more →
·3 min
Kimi K3 vs Claude Real-World Test: Near-Equal Performance at 1/8 the Price
Real-world comparison of Kimi K3 vs Claude flagship across e-commerce pages, 3D fighting games, and flight simulators. Kimi K3 delivers 90% output quality at 1/8 the price with faster speeds and local deployment support.
Read more →
·3 min
Kimi K3 Launch: A 2.8 Trillion Parameter Flagship Model Makes Its Grand Entrance
Moonshot AI launches Kimi K3 with 2.8 trillion parameters and 1M token context. Google delays Gemini 3.5 Pro, AI coding tools upgrade collectively as competition shifts to coding and Agent capabilities.
Read more →
·3 min
Kimi K3 Open-Source Release: 2.8 Trillion Parameters Tops Global Programming Leaderboard
Moonshot AI's Kimi K3 launches with 2.8 trillion parameters, tops LMArena frontend coding leaderboard as world #1, completing tasks at one-third competitors' cost. Fully open-source for commercial use.
Read more →
·4 min
Cursor Start India Plan at ₹649/Month: Decoding the Regional Pricing Strategy for AI Coding Tools
Cursor launches its Start plan for India at ₹649/month (~$7.70), featuring Grok 4.5 and Composer Agent. A deep dive into the PPP pricing strategy and its market implications.
Read more →
·3 min
Deep Dive into Inkling: The First Frontier Open-Source Multimodal LLM from the West
Thinking Machines releases Inkling, an open-source multimodal LLM with near-trillion MoE parameters, 1M token context, Apache 2.0 license. Deep dive into architecture, benchmarks, and pricing.
Read more →
·4 min
Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop
Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.
Read more →
·4 min
Laguna S 2.1 Deep Dive: Hands-On Local Deployment of a 118B Parameter Coding Model
In-depth review of Poolside's Laguna S 2.1 open-source coding model: MoE architecture, RL training, DGX Spark local deployment, and real-world agentic coding tests with 8B active parameters.
Read more →
·4 min
GPT-5.6 Value Maxing Practical Guide: From Token Stacking to Output Optimization Methodology
Deep dive into OpenAI GPT-5.6 Value Maxing strategies covering Sol/Terra/Luna model selection, KV cache optimization, Prompt compression, and programmatic tool calling to help developers achieve more output with fewer Tokens.
Read more →
·4 min
Kimi K3 Deep Dive: 2.8 Trillion Parameter Open-Source Model Closes the Gap with Closed-Source Frontier for the First Time
Moonshot AI releases Kimi K3 open-weight model with 2.8T parameters and 1M token context. Our deep dive covers coding, 3D dev, agent capabilities, and safety concerns.
Read more →
·4 min
Kimi K2/K3 Technical Speculation: Deep Dive into RL Hyperparameters and Multi-Teacher Distillation
Deep analysis of RL hyperparameter tuning challenges and 9-policy multi-teacher distillation in Kimi K2/K3 training, exploring the shift from scale to training craft.
Read more →
·2 min
Gemini Always Crashes on the 16th Turn? Error 1076 Root Cause Analysis and Workarounds
Google's Gemini consistently triggers Error 1076 on the 16th conversation turn, regardless of context size. Analysis points to a session state management defect, with three workarounds provided.
Read more →
·3 min
Apple Silicon LLM Inference Speed Benchmarks: A Complete Guide to Local Large Model Performance on Mac
In-depth analysis of Apple Silicon local LLM inference speed benchmarks covering M-series memory bandwidth, model quantization, MLX framework optimization, and Mac configuration guidance.
Read more →