NanoGPT Speedrun
An open-source community speedrunning activity aimed at training a GPT model to a specified performance benchmark in the shortest wall-clock time on fixed hardware, emphasizing algorithmic and engineering optimization efficiency rather than raw compute scaling.
Timeline (last 90 days)
The NanoGPT Speedrun concept originated from Andrej Karpathy's nanoGPT project.
In the NanoGPT Speedrun, Muon is typically applied only to the matrix weights of hidden layers, while the embedding layer and output layer retain AdamW.
NanoGPT Speedrun records must have their code made public so that anyone can verify and reproduce them.
All Facts (3)
The NanoGPT Speedrun concept originated from Andrej Karpathy's nanoGPT project.
50%UnverifiedIn the NanoGPT Speedrun, Muon is typically applied only to the matrix weights of hidden layers, while the embedding layer and output layer retain AdamW.
50%UnverifiedNanoGPT Speedrun records must have their code made public so that anyone can verify and reproduce them.
50%