Recursive Generalization
A phenomenon in which a neuron gradually expands the scope of its responses along chains of semantic relations during training. The neuron first becomes sensitive to a local feature, then generalizes through a shared relational structure to longer phrases containing that feature, and ultimately responds to semantically equivalent complete expressions.
Timeline (last 90 days)
Camerata proposes recursive generalization: generalization is not local but unfolds layer by layer like a fractal; for example, a neuron sensitive to the letter IY extends to DIY and to do-it-yourself, which does not contain IY.
Camerata proposed the concept of "recursive generalization": neurons branch and grow along related directions during training, and each branch has a provenance rather than being forcibly stuffed in as superposition.
Cammarata proposes the theory of recursive generalization, arguing that neurons gradually expand their responses along continuous dimensions of semantic or perceptual features during training, leading to apparent polysemanticity.
All Facts (3)
Camerata proposes recursive generalization: generalization is not local but unfolds layer by layer like a fractal; for example, a neuron sensitive to the letter IY extends to DIY and to do-it-yourself, which does not contain IY.
50%UnverifiedCamerata proposed the concept of "recursive generalization": neurons branch and grow along related directions during training, and each branch has a provenance rather than being forcibly stuffed in as superposition.
50%UnverifiedCammarata proposes the theory of recursive generalization, arguing that neurons gradually expand their responses along continuous dimensions of semantic or perceptual features during training, leading to apparent polysemanticity.
50%