Subspace Locking
The phenomenon in model training whereby capabilities from different domains are encoded into relatively fixed and separated subspaces of the representation space; subspaces formed early are difficult to reallocate or share across domains, reducing the utilization of the representation space.
Timeline (last 90 days)
In terms of spatial dimensions, the researchers summarized two types of subspace locking phenomena: intra-domain and cross-domain.
Subspace locking refers to the phenomenon in which the model encodes the capabilities of different domains into relatively fixed and separated subspaces in the representation space during training; once formed early on, it is difficult for subsequent training to reallocate or share them.
The concept of subspace locking draws on research on feature subspaces and task interference in representation learning, providing a geometric perspective for understanding negative transfer in multi-task learning.
All Facts (3)
In terms of spatial dimensions, the researchers summarized two types of subspace locking phenomena: intra-domain and cross-domain.
50%UnverifiedSubspace locking refers to the phenomenon in which the model encodes the capabilities of different domains into relatively fixed and separated subspaces in the representation space during training; once formed early on, it is difficult for subsequent training to reallocate or share them.
50%UnverifiedThe concept of subspace locking draws on research on feature subspaces and task interference in representation learning, providing a geometric perspective for understanding negative transfer in multi-task learning.
50%