Verbatim Repetition
The phenomenon in which the same rule or category feature is stated repeatedly in nearly identical wording within a pretraining corpus. It reinforces the model's memorization of specific literal patterns and weakens its capacity for flexible learning.
Timeline (last 90 days)
Verbatim repetition refers to the repeated statement of the same rule or category feature in nearly identical wording in the pretraining corpus, for example large amounts of text all written in fixed sentence patterns such as "birds can fly."
Verbatim repetition strengthens the model's memory of specific literal patterns, making the model tend toward rote memorization rather than inducing transferable category structures.
Empirical results show that stronger context and category–feature co-occurrence likelihood (PMI) improve learning in larger models, whereas verbatim rule-style statements weaken flexible category learning.
Controlled experiments confirm that verbatim repetition reduces flexible learning, whereas mixing multiple forms of knowledge expression yields more robust category learning.
All Facts (4)
Verbatim repetition refers to the repeated statement of the same rule or category feature in nearly identical wording in the pretraining corpus, for example large amounts of text all written in fixed sentence patterns such as "birds can fly."
50%UnverifiedVerbatim repetition strengthens the model's memory of specific literal patterns, making the model tend toward rote memorization rather than inducing transferable category structures.
50%UnverifiedEmpirical results show that stronger context and category–feature co-occurrence likelihood (PMI) improve learning in larger models, whereas verbatim rule-style statements weaken flexible category learning.
50%UnverifiedControlled experiments confirm that verbatim repetition reduces flexible learning, whereas mixing multiple forms of knowledge expression yields more robust category learning.
50%