#prefix caching
101 related articles
·3 min
Codex on Both Fronts: A Complete Setup Guide for the Desktop and CLI AI Coding Clients
A complete guide to installing and configuring OpenAI Codex desktop and CLI clients, covering model settings, API relay integration, prompt caching, and real cost data for GPT-5.6 AI coding.
Read more →
·3 min
The Truth About Codex Ultra Mode: It's Not a Reasoning Level
Deep dive into OpenAI Codex's Ultra mode: it's not a reasoning level but a system prompt switcher. Learn why Ultra burns tokens, the V1/V2 agent system issues, and how Claude Code Workflows gets it right.
Read more →
·3 min
Render Code as Images and Send to Claude — Cut API Costs by 70%
PXPipe renders system prompts and tool docs as images sent to Claude, exploiting visual token pricing. Real-world results: 68% fewer input tokens, 59–70% lower bills.
Read more →
·4 min
Two Paths to Better AI Reasoning: Test-Time Scaling and Formal Verification
This article synthesizes two MSR India Summit talks, exploring two key paths to better AI reasoning: test-time scaling with variable granularity search, and a formal verification framework for trustworthy agent execution.
Read more →
Deep Dives
Deep Dive into OpenAI's Official GPT-5…
·3 min
Deep Dive into OpenAI's Official GPT-5.6 Prompting Guide: The Shift from Manual to Automatic
A deep dive into OpenAI's official GPT-5.6 Sol prompting guide: conciseness-first, outcome-oriented design, autonomy boundaries, tool routing, and reasoning intensity tuning.
Read more →
MemStitch Zero-Copy Context Bridging: …
·2 min
MemStitch Zero-Copy Context Bridging: How It Achieves 25x TTFT Speedup in vLLM
A deep dive into how MemStitch's zero-copy context bridging achieves 25x TTFT speedup in vLLM. Covers KV Cache optimization, prefill acceleration, and practical developer value.
Read more →
GPT-5.6 Trio Launches: Luna, Terra, an…
·3 min
GPT-5.6 Trio Launches: Luna, Terra, and Sol Built for Long-Running Agentic Tasks
OpenAI officially launches the GPT-5.6 family: Luna, Terra, and Sol, with 1M token context and a focus on long-running agentic performance. A deep dive into three-tier pricing, Agents' Last Exam results, the SWE-Bench Pro controversy, and new API features like programmatic tool calling and native multi-agent support.
Read more →
·5 min
Practical AI Cost Optimization: A Deep Dive into Intelligent Routing and Context Compaction
AI bills keep soaring? This article breaks down two core cost-reduction strategies: intelligent routing via an LLM Gateway, and context compaction to cut Token usage—no major refactoring needed.
Read more →
·5 min
The Five Unsolved Challenges of Production MLOps: A Deep Dive into GPU Scheduling, Fault-Tolerant Training, and Model Observability
A deep dive into the five genuinely tough challenges of production MLOps: fault-tolerant training on Spot instances, cross-team GPU scheduling, data reproducibility, model observability, and inference cost optimization.
Read more →
·4 min
GPT-5.6 Released: Coding Benchmarks Surpass Rivals, Inference Speed 10x Faster
Just 12 days after a rival launch, OpenAI released GPT-5.6, scoring 91.9% on Terminal-Bench 2.1 to surpass competitors. Ultra mode supports multi-agent collaboration, inference hits 750 tokens/sec.
Read more →
·6 min
A Deep Dive into AI Agent Harnesses: The Core Engineering Behind Agent Frameworks
What is an AI Agent's harness? This article systematically dissects the core components of agent frameworks: context management, tool use, control loops, and caching strategies—revealing why the same model performs so differently across harnesses.
Read more →
·4 min
LingBot-World 2.0 Meets SGLang: A Deep Dive into the Evolution of Real-Time Interactive World Models
SGLang-Diffusion now officially supports LingBot-World 2.0, delivering leaps in resolution and temporal consistency. With live sessions, chunked streaming, and camera control, world models achieve low-latency controllable interaction.
Read more →
·4 min
Claude Sonnet 5 In-Depth Review: Impressive Performance, but Hidden Traps in the Pricing Strategy
A full review of Claude Sonnet 5: major agentic gains, benchmarks near Opus 4.8, but a Tokenizer switch inflates real costs, nearly erasing the price gap with Opus. We break down the pricing traps.
Read more →
·6 min
A Deep Dive into the vLLM Inference Framework: Core Principles of Throughput Optimization and Interview Guide
An in-depth analysis of the vLLM inference framework's core principles: from the meaning of throughput (tokens/s), to the bottlenecks of autoregressive generation, to KV Cache, PagedAttention, and continuous batching.
Read more →
·3 min
GPT-5.6 Deep Dive: SOL, TERRA & LUNA — Performance Benchmarks and Pricing Breakdown
OpenAI's GPT-5.6 series (SOL, TERRA, LUNA) benchmarked via PinBash: major gains in math and backend tasks, but frontend visuals remain a weakness. Full pricing and model selection guide.
Read more →
·3 min
TB/s Memory Tier Accelerates SGLang: How X-Mem Reduces TTFT by 6.7x
Netpreme integrates X-Mem™ MPU into SGLang HiCache, achieving up to 6.7× TTFT reduction and 33–50% TPS gains at 98% prefix cache hit rates. Here's the technical breakdown.
Read more →
·2 min
Deep Dive into Tencent Hunyuan Hy3: How a 295B MoE Model Cracks the Agent Deployment Challenge
Tencent Hunyuan Hy3 launches with a 295B MoE architecture activating just 21B params and 256K context. Hallucination cut from 12.5% to 5.4%, MRCR nearly doubled, with MTP and EAGLE decoding and Day-0 SGLang support.
Read more →
·2 min
GPT-5.6 Three-Model Showdown: A Complete Breakdown of So, Terra, and Luna
OpenAI's GPT-5.6 preview introduces So, Terra, and Luna. All three score perfect marks on long-horizon agentic tasks, with Terra priced 50% below GPT-5.5.
Read more →
·4 min
GPT-5.6 Three-Model Deep Dive: Sol, Terra, and Luna Benchmarked and Compared
OpenAI's GPT-5.6 series benchmarked: flagship Sol, balanced Terra, and lightweight Luna tested head-to-head. Agentic tasks rival top models, Luna starts at $1/M tokens. Full comparison with Fable 5 and Opus 4.8.
Read more →
·4 min
Kun: An Open-Source AI Coding Agent Deeply Optimized for DeepSeek
Kun is an open-source AI coding agent optimized for DeepSeek and domestic users, with nearly 5,000 GitHub stars. Features include requirements drafting, inline diffs, cost visualization, and mobile monitoring. Real-world cache hit rates reached 97%, keeping costs extremely low.
Read more →