Looped Transformer
A Transformer architecture in which the same set of network layers is called repeatedly in a loop. It inserts many serial computation steps between two text outputs, increasing hidden serial depth without adding parameters.
Timeline (last 90 days)
The Looped Transformer repeatedly calls the same set of layers in a loop, thereby inserting far more serial computation steps between two text outputs than in a conventional forward pass.
In the current paradigm, 'opaque serial depth' grows only about 1.2x per year, whereas looped transformers may bring a jump of 10x or even higher.
All Facts (2)
The Looped Transformer repeatedly calls the same set of layers in a loop, thereby inserting far more serial computation steps between two text outputs than in a conventional forward pass.
50%UnverifiedIn the current paradigm, 'opaque serial depth' grows only about 1.2x per year, whereas looped transformers may bring a jump of 10x or even higher.
50%