Unverified50% confidenceFactExact time
For autoregressive language generation, CAGD's teacher unrolling distillation can precisely decompose sequence divergence via the chain rule.
1
Sources
50%
Confidence
Long-term
Relevance
10/7/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedDiscrete masked diffusion draws on BERT's masked language modeling idea—randomly masking and restoring sequences during training and iteratively filling in from a fully masked sequence at inference—with representative works including MDLM and SEDD.73% similarUnverifiedCAGD is more flexible than hard replay, allowing the student to produce outputs on old prompts that differ in wording from the teacher but are semantically equivalent.71% similarUnverified跨分词器蒸馏的标准做法是将教师top-K logits投影到学生token上，在分段边界处把教师token映射到对应学生token序列的第一个token，这是一种多对一映射69% similarUnverified非自回归机器翻译通过知识蒸馏解耦策略使用自回归教师模型生成的伪标签训练，将BLEU分数提升幅度超过10个点，同时推理保持并行解码65% similarUnverifiedCommon directions for improving pseudo-label quality include confidence filtering, introducing language model rescoring (LM rescoring), and ensembling multiple teacher models.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/982973API
curl https://kongchang.com/api/v1/knowledge/claims/982973MCP
get_claim(id=982973)