Local Interaction

An attention mechanism that performs interaction only within local windows or neighborhoods. Its computational complexity is typically linear in the number of tokens, and it is used to model local details efficiently.