Unverified50% confidenceFactExact time
A cache hit requires the prefix to be completely identical and to exceed a certain length threshold (typically 1,024 tokens).
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe official recommendation is to appropriately relax the max tokens limit to leave room for both reasoning and responses, avoiding truncated output.69% similarUnverifiedChunk size is typically recommended to be between 256 and 512 tokens, with a 10–15% overlap region preserved.68% similarUnverifiedT5 系列最大支持 512 个 Token 的长序列，而 CLIP 文本编码器最大序列长度通常仅为 77 个 Token66% similarVerifiedTaking the LLaMA-2 70B model as an example, a sequence of 2048 tokens requires approximately 4 GB of VRAM for the KV cache alone.66% similarUnverifiedIn practical engineering, a chunk_size between 256 and 1024 tokens and a chunk_overlap between 20 and 100 tokens are fairly common empirical ranges.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/996114API
curl https://kongchang.com/api/v1/knowledge/claims/996114MCP
get_claim(id=996114)