Weight tying sets the input embedding matrix and the output projection matrix as the same set of parameters; early models such as GPT-2 have already adopted this approach, which reduces parameters and helps training stability.