Unverified50% confidenceFactExact time
The study validates on three datasets—TinyStories, WikiText-103, and CNN/DailyMail—scaling each corpus from 512 chunks to 2,048 chunks, with each chunk consisting of 64 tokens.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe attention matrix size required to process a 128K token sequence is 256 times that of an 8K token sequence.66% similarUnverifiedWhen performing text classification on CPU, a chunk size of 256 or 512 tokens generally outperforms an 8192 large window.65% similarUnverifiedBase64 编码将每3字节二进制数据拆分为4组6位，输出长度约为原始数据的4/3倍65% similarUnverifiedKV Cache stores the Key and Value matrices of all previous tokens during the Decode phase, grows with conversation turns and context window size, and can occupy anywhere from a few GB to tens of GB of memory in long-text scenarios64% similarUnverifiedSHADOW writes attention states to disk at 1-bit precision, consuming 288 bytes per token, with the index occupying only 22 bytes per token, record lookup taking approximately 1 microsecond, and attention states being reinjected into the model in approximately 0.03 ms, without using a vector database or embedding model.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1001474API
curl https://kongchang.com/api/v1/knowledge/claims/1001474MCP
get_claim(id=1001474)