Unverified50% confidenceFactExact time
A model can have on the order of O(N²) parameters yet carry only on the order of O(N) cross-time state, creating a mismatch between capacity and memory.
1
Sources
50%
Confidence
Long-term
Relevance
10/6/2026
First Seen
Sources
Related Entities
Related Claims
Unverified对于千亿参数模型，一次完整检查点保存可能涉及数TB数据，包含模型权重和Adam优化器的一阶及二阶矩状态71% similarVerifiedThe inference stage of large models is memory-bound (limited by memory bandwidth); generating each token requires reading the entire model weights from VRAM, resulting in extremely low computational intensity.69% similarUnverified自注意力机制的QK^T计算对于序列长度n、隐藏维度d的模型需要O(n²d)次乘加运算68% similarUnverifiedThe degree of model memorization is highly correlated with the number of data repetitions; when the number of parameters greatly exceeds the information entropy of the training data, the model tends to memorize rather than generalize.67% similarUnverifiedN-gram统计语言模型通过相邻词的条件概率捕捉局部上下文，受限于固定窗口大小，无法处理长距离依赖，参数量随词汇表指数膨胀66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/981668API
curl https://kongchang.com/api/v1/knowledge/claims/981668MCP
get_claim(id=981668)