Multi-Head Mechanism

A mechanism that projects queries, keys, and values into multiple low-dimensional subspaces and computes attention in parallel, allowing the model to attend to different types of relationships simultaneously. It is one of the core designs of the Transformer.