LoopLMs
A class of language model architectures that reuse shared parameters across multiple recursive computation steps. These models deepen reasoning capability without substantially increasing parameter count and inherently support test-time compute scaling.
Timeline (last 90 days)
The architectural inspiration for Looped Language Models comes from earlier works such as the Universal Transformer and DEQ (Deep Equilibrium Model).
A 12-layer LoopLM can simulate a computational depth equivalent to 48 layers, because the parameters are shared and reused in each recursion.
The fundamental challenge in post-training LoopLMs is that shared parameters mean a gradient update for any given loop depth simultaneously affects the behavior of all depths, making inter-layer interference effects prone to occur.
All Facts (3)
The architectural inspiration for Looped Language Models comes from earlier works such as the Universal Transformer and DEQ (Deep Equilibrium Model).
50%UnverifiedA 12-layer LoopLM can simulate a computational depth equivalent to 48 layers, because the parameters are shared and reused in each recursion.
50%UnverifiedThe fundamental challenge in post-training LoopLMs is that shared parameters mean a gradient update for any given loop depth simultaneously affects the behavior of all depths, making inter-layer interference effects prone to occur.
50%