Large language models learn the syntax rules of programming languages and the mapping between natural language and code by pre-training on billions of lines of open-source code on platforms such as GitHub.