Mainstream LLMs are based on the Transformer architecture proposed by Google in 2017, whose core innovation is the Self-Attention mechanism.