Model Collapse
Model collapse is a phenomenon observed when a generative model is repeatedly trained on data it has itself generated. Over successive iterations, the output distribution degrades progressively: low-probability events are systematically forgotten and outputs converge toward a narrow, homogeneous set, resulting in a loss of diversity and fidelity relative to the original data distribution.
Core Facts
When language models are repeatedly trained on data generated by themselves or similar models, systematic degradation occurs, with the diversity of model outputs decreasing with each generation and the tail information of the distribution gradually being lost.
During model collapse, the model successively experiences a reduction in output diversity, loss of minority categories and edge knowledge, and ultimately produces meaningless or highly repetitive outputs.
A 2023 study by Oxford University and other institutions showed that training AI models on data containing large amounts of AI-generated content causes them to lose the ability to capture the tails of the data distribution, leading to homogenization of outputs and quality degradation, known as model autophagy.
Timeline (last 90 days)
Since 2023, as AI-generated text has flooded the internet at scale, model collapse has shifted from a theoretical concern to an observable real-world threat.
When language models are repeatedly trained on data generated by themselves or similar models, systematic degradation occurs, with the diversity of model outputs decreasing with each generation and the tail information of the distribution gradually being lost.
During model collapse, the model successively experiences a reduction in output diversity, loss of minority categories and edge knowledge, and ultimately produces meaningless or highly repetitive outputs.
A 2023 study by Oxford University and other institutions showed that training AI models on data containing large amounts of AI-generated content causes them to lose the ability to capture the tails of the data distribution, leading to homogenization of outputs and quality degradation, known as model autophagy.
Research published in 2023, such as "The Curse of Recursion," shows that when models are repeatedly trained on their own generated data, the data distribution gradually degrades, diversity continuously decreases, and model capabilities systematically decline (model collapse).
模型坍缩的核心机制是后续模型在AI生成数据上训练时逐渐丢失原始人类数据中的长尾分布信息，导致输出多样性收窄
Model Collapse was first systematically described by machine learning researcher Ilia Shumailov and colleagues in 2023 in the journal Nature.
All Facts (8)
When language models are repeatedly trained on data generated by themselves or similar models, systematic degradation occurs, with the diversity of model outputs decreasing with each generation and the tail information of the distribution gradually being lost.
75%VerifiedDuring model collapse, the model successively experiences a reduction in output diversity, loss of minority categories and edge knowledge, and ultimately produces meaningless or highly repetitive outputs.
65%VerifiedA 2023 study by Oxford University and other institutions showed that training AI models on data containing large amounts of AI-generated content causes them to lose the ability to capture the tails of the data distribution, leading to homogenization of outputs and quality degradation, known as model autophagy.
65%Verified模型塌陷（Model Collapse）指未来模型若大量在AI生成内容上训练，将逐渐失去人类写作多样性并陷入退化循环
65%UnverifiedResearch published in 2023, such as "The Curse of Recursion," shows that when models are repeatedly trained on their own generated data, the data distribution gradually degrades, diversity continuously decreases, and model capabilities systematically decline (model collapse).
70%UnverifiedModel Collapse was first systematically described by machine learning researcher Ilia Shumailov and colleagues in 2023 in the journal Nature.
60%UnverifiedSince 2023, as AI-generated text has flooded the internet at scale, model collapse has shifted from a theoretical concern to an observable real-world threat.
50%Unverified模型坍缩的核心机制是后续模型在AI生成数据上训练时逐渐丢失原始人类数据中的长尾分布信息，导致输出多样性收窄
50%