Register Token

A mechanism proposed in Vision Transformer research that inserts special tokens into the input sequence that do not correspond to any actual image region. These tokens provide the model with dedicated space for processing global information, addressing the issue of abnormally high activation values observed in certain patch tokens in ViT.