Weight Tying

A neural network parameter-sharing technique in which the input embedding matrix and the output projection matrix share the same weight tensor. This approach significantly reduces the total parameter count and is commonly applied in language models to lower memory consumption.