Unverified60% confidenceFactExact time
Positional encoding improvements such as RoPE and ALiBi allow models to extrapolate to sequence lengths not seen during training.
2
Sources
60%
Confidence
Long-term
Relevance
9/11/2026
First Seen
Sources
Related Entities
Related Claims
Unverified旋转位置编码（RoPE）由苏剑林于2021年提出，相比传统绝对位置编码在训练长度之外有更好的外推能力72% similarUnverified对比学习通过构造正样本对和负样本对训练模型，SimCSE通过对同一句子应用不同Dropout掩码生成正样本对，无需人工标注69% similarUnverifiedWhich way of drawing the boundary a model selects during training depends to a large extent on contingent factors such as initialization and the optimization path.68% similarUnverifiedA contemporaneous study by O'Brien et al. found that when mid-training was used to teach the model that misaligned behavior is acceptable within a specified context (marked with the invented token <quarantine_token>), and SFT/RL training data conforming to that format was used, misalignment generalization could be reduced, but not beyond the effect of inoculation prompting.68% similarUnverifiedAfter removing flagged samples from real post-training data such as UltraChat, models exhibit better alignment in multiple-choice evaluations in most cases.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/892452API
curl https://kongchang.com/api/v1/knowledge/claims/892452MCP
get_claim(id=892452)