1 related articles
Experiments show PBT doesn't accelerate grokking but helps models escape a no-solution hyperparameter dead zone. On the mod 97 addition task, fast PBT escapes the dead zone while fixed configurations all fail. Schedule replay verifies that the result is carried by the hyperparameter trajectory.
