Unverified50% confidenceFactExact time
The model window promised support for about one million tokens, but the underlying KV pool can actually hold only 681,280 tokens.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/31/2026
First Seen
Valid until: 1/29/2027
Sources
Cloudflare 200 OK Leak: 155 Responses Crossed User Data
youtubeSwarmAgent10/27/2026
Related Entities
Related Claims
VerifiedFor a typical 7B-parameter model, the KV Cache per 1K tokens requires approximately 0.5–1 GB of VRAM.77% similarUnverifiedMost models have a single output limit of between 4,000 and 8,000 tokens, with some high-end models supporting up to 32,000 tokens.75% similarVerifiedTaking the LLaMA-2 70B model as an example, a sequence of 2048 tokens requires approximately 4 GB of VRAM for the KV cache alone.71% similarUnverifiedWhen running the Llama 3.1 8B model on an i7-100, SGLang reaches about 16200 tokens/s and vLLM about 12500 tokens/s, a difference of about 30%.70% similarUnverifiedIn the experiments, the input averages only 3.1K tokens, even though the context window exceeds 100K tokens.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1119350API
curl https://kongchang.com/api/v1/knowledge/claims/1119350MCP
get_claim(id=1119350)