The fundamental blind spot of online distillation is that states the teacher can reach but the student rarely or never reaches receive almost no supervision signal, because the student's extremely low sampling probability leaves these regions unable to obtain gradients.