Unverified50% confidenceBenchmarkExact time
将学生词表扩大至4096后，熵保留率不升反降，小幅滑落至13%
1
Sources
50%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
跨分词器知识蒸馏：85%信息丢失的根因与链式法则解法
Related Claims
Unverified在标准投影方案下，教师分布的熵从2.09 bits骤降至0.32 bits，仅约15%的教师信息得以保留69% similarUnverified将年龄字段从100个可能值泛化为5个区间，信息熵从约6.64比特降至约2.32比特，信息量损失超过60%65% similarUnverified投机解码在代码生成等低熵任务中接受率可达70-85%，在创意写作等高熵任务中可能降至40-60%60% similarUnverifiedFinite-sample estimation of entropy rate faces a systematic negative bias problem, where plug-in estimators tend to underestimate the true entropy; this can be compensated using the Miller-Madow correction by adding a (k-1)/(2N) term.60% similarPartially VerifiedThe more certain a distribution is, the lower its entropy and the shorter the required encoding; the more random a distribution is (such as a uniform distribution), the higher its entropy.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/508981API
curl https://kongchang.com/api/v1/knowledge/claims/508981MCP
get_claim(id=508981)