Unverified50% confidenceFactExact time
Anthropic要求缓存的最小单位是1024个Token，OpenAI以128个Token为粒度进行缓存匹配
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/15/2026
First Seen
Valid until: 11/13/2026
Sources
Related Claims
UnverifiedAnthropic's Claude requires the prefix length to exceed 1,024 tokens before triggering Prompt Caching, while OpenAI automatically caches in granularities of 128 tokens.69% similarUnverifiedTaking the LLaMA-2 70B model as an example, a sequence of 2048 tokens requires approximately 4 GB of VRAM for the KV cache alone.67% similarUnverifiedThe interrupt vector table is located in the lowest 1,024 bytes of memory and stores the entry addresses of 256 interrupt handlers.67% similarUnverified单个70B参数模型可能占用40GB以上显存，单张A100仅能承载极有限的并发请求64% similarUnverifiedFor 1 million tokens, the KV Cache alone can consume hundreds of GB of VRAM at FP16 precision, far exceeding the 80 GB capacity limit of a single A100/H100.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/753257API
curl https://kongchang.com/api/v1/knowledge/claims/753257MCP
get_claim(id=753257)