Unverified50% confidenceTradeoffExact time
Under the continuation protocol, each continuation resends the prompt plus the already-decoded content, so the cumulative input token count grows far faster than the context.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedAPI billing covers both input tokens and output tokens, and as a conversation extends, the input tokens carried in each request grow linearly.74% similarVerifiedContinuous Batching adopts iteration-level scheduling granularity, allowing completed requests to be removed and new requests to be inserted upon each token generation completion.72% similarUnverifiedAutomatic prompt tuning consumes a large number of tokens in the model's back-and-forth self-correction process, incurring additional API call, token, and latency overhead.71% similarUnverifiedA prompt cache hit requires the request prefix to be byte-for-byte identical to what was written at cache time; mixing dynamic content such as timestamps or session IDs into system prompts will cause frequent cache invalidation.68% similarUnverifiedTo maximize cache hit rate, stable content (system prompts, file contents) should be placed at the prefix of the prompt, with dynamic content appended at the end68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1021147API
curl https://kongchang.com/api/v1/knowledge/claims/1021147MCP
get_claim(id=1021147)