Unverified50% confidenceFactExact time
In the replaced model, the tied embedding/lm_head matrix remains bf16, so each token actually reads about 1.263 GB of weights (plus about 0.06 GB of KV cache); calculated by actual bytes moved, 25.5 tok/s corresponds to about 34 GB/s, roughly 10% of measured bandwidth.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/12/2026
First Seen
Valid until: 10/26/2026
Sources
Related Entities
Related Claims
UnverifiedWhen the model loads, it first allocates a full-precision shell of approximately 1.29 GB and then reads in 643 MB of full weights, with peak memory potentially reaching approximately 3,400 MB.77% similarUnverifiedMotrix's actual memory usage is in the range of 150–250 MB.75% similarUnverifiedIf all 36 layers of Spark X 2.5 tracked the full text, the KV cache overhead for tracking 1 million tokens would be approximately 155 GB, while the sparse design compresses this to approximately 39 GB74% similarUnverified量化可将700亿参数模型的存储体积从FP16的约140GB压缩至INT4的约35-40GB，能在单张RTX 4090配合CPU内存卸载的消费级硬件上运行74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1006844API
curl https://kongchang.com/api/v1/knowledge/claims/1006844MCP
get_claim(id=1006844)