Unverified50% confidenceFactExact time
Traditional soft-label distillation requires the draft model to approximate the target model's full probability distribution at every position, including high-entropy regions, which is inconsistent with the goal of maximizing acceptance length at inference time.
1
Sources
50%
Confidence
Long-term
Relevance
10/22/2026
First Seen
Sources
Related Entities
Related Claims
Unverified跨分词器蒸馏前应先测量熵保留率或概率质量对齐度以避免在错误目标上浪费算力73% similarUnverified标准交叉熵损失在目标占图像面积较小时会导致严重的类别不平衡问题，模型倾向于全部预测为背景73% similarUnverifiedThe paper designs an explicit target density that lowers probability mass near undesirable conditions, with the strength of the suppression controlled by a parameter.72% similarUnverifiedWhen a model encounters out-of-distribution inputs, prediction entropy increases, the softmax output tends toward a uniform distribution, and the range of candidate tokens permitted by nucleus sampling expands, giving long-tail anomalous tokens a chance of being sampled.72% similarUnverifiedAdaptiveSpec's stepwise margin rule: when the ratio of the target model's probability on a draft token to its top-1 token probability exceeds a certain threshold, the draft token is accepted even if the two do not exactly match.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1084552API
curl https://kongchang.com/api/v1/knowledge/claims/1084552MCP
get_claim(id=1084552)