CodeBERT

CodeBERT is a bimodal pre-trained language model developed by Microsoft Research, based on the Transformer architecture and pre-trained jointly on natural language and multiple programming languages. The model supports joint semantic modeling of natural language and code, enabling tasks such as code search, code documentation generation, defect detection, and code understanding. It serves as a foundational pre-trained model for programming language processing.