Universal Transformer uses weight tying to allow all recurrent steps to reuse the same set of parameters, increasing computational depth without increasing the parameter count.