BERT uses an encoder architecture, the GPT series uses a decoder architecture, and T5 and BART use an encoder-decoder architecture