Unverified50% confidenceFactExact time
The cross-entropy loss optimized during modern large language model training is essentially minimizing the encoding length of text.
1
Sources
50%
Confidence
Long-term
Relevance
9/22/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen large language models are trained on text generated by other AI, the readability of the output declines.77% similarUnverifiedLarge language models exhibit an 'instruction drift' phenomenon, where the model's probability of following early instructions gradually decreases as conversation turns increase and the context window is filled with large amounts of code.74% similarUnverifiedCode-mixing phenomena can cause significant performance degradation in existing pre-trained language models.74% similarUnverified当训练数据中某些文本出现频率极高时，大语言模型可能产生记忆化（Memorization）现象，能够近乎逐字输出特定段落73% similarUnverifiedTraditional n-gram models and neural language models trained on a single language assign extremely low probability scores to cross-lingual word sequences, causing the decoder to tend to forcibly map foreign-language words to phonetically similar words in the target language.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/940780API
curl https://kongchang.com/api/v1/knowledge/claims/940780MCP
get_claim(id=940780)