[KongchangAI]
Concept递归自训练 / self-training / 递归自微调

Self-Training

A training paradigm in which a language model, over multiple generations of iteration, fine-tunes on its own outputs as training data; it is often used to study model collapse and distribution shift.

Timeline (last 90 days)

Nov 9

In self-training, the training temperature affects the sampling distribution during data generation: temperature T adjusts the model's output probabilities via p_i ∝ exp(z_i/T), with T>1 making the distribution flatter and T<1 making it sharper.

Unverified50%
Nov 9

The study uses English GPT-2 for nine generations of self-training, fine-tuning each generation on the previous generation's outputs and varying the sampling temperature of the training data; all models are measured with the same fixed sampler to ensure fair comparison.

Unverified50%

All Facts (2)

Source Articles