Expert iteration is a training paradigm between imitation learning and reinforcement learning: first let the current model generate a large number of trajectories, use a reward function to pick out the best-performing ones, then supervised-fine-tune on these expert samples, and repeat the cycle; this paradigm has no value function, no policy gradient, and no negative penalty for failed samples, and the model only learns to imitate its own highlight moments, making reward hacking more likely.