Unverified50% confidenceFactExact time
When the concatenation order of the cache is reversed, the single-token attention test still passes, but decoding multiple new tokens together fails.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhether prompt caching hits depends on byte-for-byte consistency of the request prefix; if even one character at the beginning differs, the entire cache is invalidated.73% similarUnverifiedThe decoder-only architecture uses a causal attention mask to ensure each token can only attend to its preceding context, making it suitable for autoregressive text generation tasks.72% similarUnverifiedPrompt caching can avoid repeated charges for identical prefixes, because system prompts and retrieved document fragments tend to be highly repetitive across multiple conversation turns.66% similarUnverifiedIn-context conditioning concatenates the text condition and image tokens into the same sequence, eliminating the cross-attention module, but it lengthens the sequence and increases the amount of attention computation.66% similarUnverifiedIf most of the prompt is repeated and fixed while cache_read_input_tokens stays near zero for a long time, it can be almost certain that the prefix changed between requests.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1060683API
curl https://kongchang.com/api/v1/knowledge/claims/1060683MCP
get_claim(id=1060683)