Representation Reorganization
A phenomenon in which the internal representation space structure of a neural network undergoes systematic adjustment when task objectives change, manifested as rapid reversal of dimension importance rankings. It is distinguished from catastrophic forgetting.
Timeline (last 90 days)
Representational reorganization refers to the phenomenon in which the structure of the network's internal representation space undergoes systematic adjustment when the task objective changes, typically manifesting as rapid flips in the ranking of dimension importance; it is distinct from catastrophic forgetting, which emphasizes the loss of performance on old tasks.
Observing representational reorganization requires measurement tools that can align representation spaces across training stages, and intrinsic metrics such as Bures geometry have advantages over neuron-by-neuron comparison.
All Facts (2)
Representational reorganization refers to the phenomenon in which the structure of the network's internal representation space undergoes systematic adjustment when the task objective changes, typically manifesting as rapid flips in the ranking of dimension importance; it is distinct from catastrophic forgetting, which emphasizes the loss of performance on old tasks.
50%UnverifiedObserving representational reorganization requires measurement tools that can align representation spaces across training stages, and intrinsic metrics such as Bures geometry have advantages over neuron-by-neuron comparison.
50%