Unverified50% confidenceSolutionExact time
存储教师top-K分布时以span内已生成的字节为条件进行条件化处理，可将熵保留率从15%提升至83-86%
1
Sources
50%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
跨分词器知识蒸馏：85%信息丢失的根因与链式法则解法
Related Claims
Unverified跨分词器蒸馏前应先测量熵保留率或概率质量对齐度以避免在错误目标上浪费算力63% similarUnverified从信息论角度看，LLM的输出熵相对较低，倾向于收敛到训练数据的高概率区域62% similarUnverifiedFor a stationary ergodic process, the entropy rate limit exists and equals the conditional entropy limit lim H(X_n|X_{n-1},...,X_1).58% similarUnverified在推理阶段内存带宽往往比纯算力更早成为瓶颈，注意力机制需在每步生成时读取全部KV缓存，上下文长度增大时内存读写开销呈平方级增长56% similarUnverifiedLarger batch sizes typically require higher learning rates (the linear scaling rule).55% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/508985API
curl https://kongchang.com/api/v1/knowledge/claims/508985MCP
get_claim(id=508985)