#KV Cache optimization
408 related articles
·7 min
Reverse-Engineering CUDA-checkpoint: The Technical Principles and Practice of Cracking GPU Cold Starts
An in-depth analysis of reverse-engineering Nvidia CUDA-checkpoint to accelerate GPU cold starts. Covers checkpoint/restore, Serverless GPU prospects, and VRAM snapshot challenges.
Read more →
·3 min
GPT-5.6 Revealed: What a 54% Token Efficiency Boost in Agentic Coding Actually Means
Sam Altman revealed GPT-5.6 boosts token efficiency 54% in agentic coding — meaning near-halved API costs, longer work chains, and faster responses. A deep dive into the technical implications for developers.
Read more →
·7 min
GPT-5.6 In-Depth Hands-On Review: How Do Soul, Tara, and Luna Really Perform?
OpenAI releases GPT-5.6 preview with three models: flagship Soul, balanced Tara, and lightweight Luna. Based on real KingBench 3 testing, this article breaks down each model's performance on math, front-end, and agentic tasks, and compares them with Anthropic Fable.
Read more →
·3 min
GPT-5.6 Three-Model Architecture Explained: Sol/Terra/Luna Pricing and Performance Overview
GPT-5.6 introduces a Sol/Terra/Luna three-model architecture. Terra delivers near-flagship performance at half the price, with new Max and Ultra reasoning tiers.
Read more →
·6 min
Grok 4.5 Deep Dive: Coding on Par with Top Models, at a Fraction of the Cost
A senior developer's 24-hour deep test of Grok 4.5: a 1.5T-param MoE model at $2/M input tokens, with coding benchmarks rivaling GPT-5.5. Real performance, token efficiency, and limits explained.
Read more →
·7 min
GLM-5.2 Released: Open-Source Coding Capability Approaches Closed-Source Flagships for the First Time
GLM-5.2 tops open-weight models in coding with a 74.4 Frontiers-WE score, beating GPT-5.5. Its MIT license enables local deployment, and the gap with closed-source flagships is closing fast.
Read more →
·4 min
FTPO: A Training-Level Solution to AI Repetition Loops
FTPO (Final Token Preference Optimization) tackles AI "Doom Loops" at the training level rather than patching them at inference time — exploring its principles, value for small/quantized models, and open challenges.
Read more →
·5 min
Is AI Inference Really Profitable? Deconstructing the Cost Economics of Inference vs. Training
Are AI companies really losing money? This article deconstructs the fundamental differences between training and inference costs, revealing why pure AI inference services have structural profitability.
Read more →
·6 min
Unsloth Major Update: DeepSeek-V4 Support and a Deep Dive into NVFP4 Quantized Export
Unsloth v0.1.481-beta adds full DeepSeek-V4-Flash support, NVFP4/FP8/imatrix GGUF quantized export, 1.3x faster GRPO, 3-5x faster MoE training, and an OpenAI-compatible API service in Studio.
Read more →
·7 min
The Agent Engineer Transition Roadmap: A Deep Dive into Three Core Skill Systems
Want to become an Agent engineer? This article systematically covers three core skill tracks—LLM fundamentals, LangChain architecture development, and enterprise deployment—to help you avoid detours.
Read more →
·6 min
GPT-5.6 Three-Model Launch: Soul/Terra/Luna Tiered Strategy and the AI National Security Game
OpenAI previews the GPT-5.6 series — Soul, Terra, and Luna — with a massive 1.5M-token context. In-depth analysis of coding leaps, the Fable 5 national security game, the heating U.S.-China AI race, and workflow economics.
Read more →
·6 min
Xiaomi XiaoAI 10.1-inch Smart Control Panel: Is the AI LLM Integration Worth It?
Xiaomi XiaoAI 10.1-inch Smart Control Panel features AI LLM Q&A, WeChat calling, and whole-home Mi IoT control. Priced at 839 yuan, ~679 yuan after national subsidy. An in-depth review of AI capabilities, screen experience, and smart home integration.
Read more →
·7 min
LangChain Hands-On Guide: From LLM Calls to Agent Development
Master LangChain from scratch: the three limitations of LLMs, init_chat_model unified interface config, the Message type system, and the path from LLM calls to Agent development.
Read more →
·6 min
AMD Ryzen AI Halo Dev Kit: A $4,000 Local LLM Powerhouse Taking on Apple and NVIDIA
AMD officially unveils the Ryzen AI Halo local AI dev kit, priced around $4,000 with 128GB unified memory, capable of running 70B LLMs locally. An in-depth look at its specs, pricing, and market competition.
Read more →
·5 min
DeepSeek DSpec Hands-On: How Speculative Decoding Accelerates LLM Inference by 6x
Hands-on report on DeepSeek's open-source inference acceleration toolkit DSpec: draft model + smart scheduling delivers lossless speedup, hitting acceptance length 6 on GSM8K and reproducing official data.
Read more →
·4 min
Career Switch to LLM Development: A Four-Level Skill Progression Roadmap
Want to switch careers into LLM development but don't know where to start? This guide breaks down a four-level skill roadmap — from basics and API calls to RAG, fine-tuning, Agent development, and multimodal — to help you build real AI career value.
Read more →
·4 min
AI Agent: A Complete 6-Week Learning Roadmap from Beginner to Production
A complete 6-week AI Agent learning roadmap covering core architecture (planning/memory/tool use), the ReAct paradigm, multi-agent collaboration, RAG integration, and production deployment.
Read more →
·6 min
Doubao and Qwen Retire AI Agents: How Compute Costs Are Reshaping AI's Commercial Boundaries
Doubao and Qwen have retired their AI Agent features. The real reason isn't regulation—it's that companion-chat users don't pay, making compute costs unrecoverable. A deep dive into AI's cost dilemma.
Read more →
·3 min
GPT-5.6 Sol Ultra Rumors Decoded: How Codex's Programming Capabilities Could Evolve
Rumors of GPT-5.6 Sol Ultra integrating into Codex are circulating in tech communities, but OpenAI hasn't confirmed anything. We analyze the real trends: context understanding, reasoning, and agentic workflows.
Read more →
·8 min
Fable 5 Opens to All Paid Users: How AI Storytelling Tools Are Reshaping Content Production
Fable 5, an AI storytelling platform, opens to all paid users and sparks debate on Hacker News. We analyze AI creation tools' practicalization trend across product positioning, competition, and access strategy.
Read more →