Teacher Forcing
A core training technique for sequence generation models, proposed by Williams and Zipser in 1989. During training, ground-truth target tokens are used as decoder inputs instead of the model's own predictions, which stabilizes gradient signals but introduces a discrepancy between training and inference distributions.
Timeline (last 90 days)
During the training phase, the decoder can use teacher forcing to process the entire target sequence in parallel, whereas during inference the model can only generate output autoregressively one token at a time.
Teacher forcing introduces the exposure bias problem: during training the real target token is fed as input, while during inference the model's own predicted token is used, and prediction errors accumulate and amplify in subsequent steps.
Teacher Forcing was proposed by Williams and Zipser in 1989; when training the decoder, the corresponding token from the true target sequence is used as input at each step rather than the model's own prediction.
All Facts (3)
During the training phase, the decoder can use teacher forcing to process the entire target sequence in parallel, whereas during inference the model can only generate output autoregressively one token at a time.
50%UnverifiedTeacher forcing introduces the exposure bias problem: during training the real target token is fed as input, while during inference the model's own predicted token is used, and prediction errors accumulate and amplify in subsequent steps.
50%UnverifiedTeacher Forcing was proposed by Williams and Zipser in 1989; when training the decoder, the corresponding token from the true target sequence is used as input at each step rather than the model's own prediction.
50%