#KV Cache optimization
399 related articles
·6 min
AMD MI355X Running GLM5.2: Inference Costs at Half of NVIDIA Blackwell
AMD MI355X achieves 2,626 tokens/sec throughput running GLM5.2 at half the TCO of NVIDIA Blackwell. Deep analysis of the technical logic, ROCm ecosystem progress, and market implications.
Read more →
·4 min
NVFP4 Quantization in Practice: A Guide to Compressing Large Models with NVIDIA Model Optimizer
Deep dive into NVFP4 quantization: using NVIDIA Model Optimizer to compress Nemotron 3 Ultra to FP4 checkpoints, reducing memory by 75% and boosting inference throughput on Blackwell GPUs.
Read more →
·3 min
DeepSeek Open-Sources DiSpark: Up to 85% Faster AI Inference Without Changing Hardware
DeepSeek and Peking University release DiSpark, an open-source framework that speeds up LLM inference by 60–85% using speculative decoding and hierarchical verification — no hardware changes or retraining required.
Read more →
·4 min
Making Any LLM Understand Video: Claude-real-video Technical Deep Dive
A deep dive into Claude-real-video: how keyframe extraction, image captioning, and ASR convert video into structured LLM-readable input for model-agnostic video understanding.
Read more →
·5 min
Claude Sonnet 5 Integrated with Devin: Enhanced Coding Performance with 30% Lower Quota Consumption
Anthropic's Claude Sonnet 5 launches on Devin Desktop and CLI, delivering frontier-level coding performance while reducing quota consumption by ~30% compared to the previous generation.
Read more →
·5 min
NVIDIA AI-Q Blueprint × OCI: A Practical Guide to Production Deployment of Enterprise AI Agents
Deep dive into NVIDIA AI-Q Blueprint production deployment on Oracle Cloud Infrastructure, covering NIM microservices, RAG architecture, multi-agent orchestration, and OCI GPU selection.
Read more →
·5 min
NVIDIA AI-Q Blueprint × OCI: A Practical Guide to Production Deployment of Enterprise AI Agents
Deep dive into NVIDIA AI-Q Blueprint production deployment on Oracle Cloud Infrastructure, covering NIM microservices, RAG architecture, multi-agent orchestration, and OCI GPU selection for enterprise AI agents.
Read more →
·3 min
Ornith 9B Tested: Can a 16GB Mac Mini Handle Local AI Coding?
Tested Ornith 9B on a 16GB M4 Mac Mini: LM Studio setup, tower defense game vs. 35B, and honest insights into small-model accuracy limits for local AI coding.
Read more →
·7 min
Harness Engineering: Core Methodology and Practical Guide for Enterprise-Level AI Programming
Can AI really replace programmers? This article explains Harness Engineering principles and its three evolutionary stages, revealing real pain points of enterprise AI programming.
Read more →
·6 min
AI Workspace Session and Cache Leaks: A Complete Analysis of Multi-Tenant Isolation Security Risks
AI workspaces face cross-tenant session and cache leak risks that can expose sensitive enterprise data. This article analyzes multi-tenant isolation pitfalls, common architectural flaws, and actionable defenses.
Read more →
·7 min
OpenAI Codex Shortcuts Getting an Upgrade — How Will the AI Programming Experience Evolve?
OpenAI announces Codex shortcut upgrades focused on developer workflow optimization. Analysis of upgrade directions, industry competition, and expected improvements to code completion and natural language triggers.
Read more →
·6 min
14x Speedup: How Manticore Rebuilt the ONNX Embedding Inference Path
Manticore Search restructured its ONNX inference path to achieve 14x faster text embeddings. Deep dive into batching, session reuse, zero-copy memory, and thread tuning for vector search systems.
Read more →
·3 min
CodeBurn: The 8.4K-Star Open-Source Tool That Tracks Every Cent of Your AI Coding Spend
CodeBurn is a privacy-first local tool for analyzing AI coding costs. It supports 31 tools including Claude Code, Cursor, and Codex, breaking down spending by model, project, and task to eliminate hidden waste.
Read more →
·6 min
A 284B Model Running on a MacBook: The Real Breakthrough and Barriers of Local DeepSeek Inference
Redis creator runs 284B-parameter DeepSeek model on a MacBook Pro at 26 tokens/sec using a pure C engine, asymmetric quantization, and MoE architecture.
Read more →
·3 min
Vibe Coding Guide: Building Web Apps with AI from Zero Experience
Vibe Coding lets non-programmers build web apps using natural language and AI agents. Learn Claude Code, model selection, cost control via cache optimization, and agentic engineering.
Read more →
·3 min
Running Local LLMs on AMD GPUs: Ditching Ollama for LM Studio Delivers 7x Speed Boost
AMD GPU black screens running local LLMs? This post-mortem covers Ollama's 3 fatal flaws and how switching to LM Studio boosted token speed from 5 to 36, with ROCm setup, Speculative Decoding, and GFX version tips.
Read more →
·2 min
AI Large Model Development Learning Path: A Complete Guide from Zero to Enterprise Projects
A complete learning roadmap for AI large model development — covering Transformer, Prompt Engineering, RAG, LangChain, Agent development, fine-tuning, and deployment.
Read more →
OpenAI and Broadcom Unveil Jalapeño Ch…
·2 min
OpenAI and Broadcom Unveil Jalapeño Chip: Built Exclusively for LLM Inference
OpenAI and Broadcom unveil Jalapeño, a custom ASIC designed for LLM inference. A deep dive into its technical logic, strategic intent, and impact on NVIDIA and the AI compute landscape.
Read more →
·7 min
AI Agent Development: A Four-Stage Competency Model from 15K to 40K RMB Salary Growth Path
A detailed four-stage competency model for AI Agent development: from Python/RAG basics (15K) to workflow orchestration (20K), inference optimization (30K), and Agent cluster governance (40K RMB).
Read more →
·7 min
AI Agent Code Execution Sandbox Selection Guide: In-Depth Comparison of E2B and Four Other Solutions
In-depth comparison of five AI Agent code execution sandbox solutions—E2B, Daytona, Modal, Cloudflare Sandbox, and Vercel Sandbox—across isolation, cold start latency, state management, and pricing.
Read more →