[KongchangAI]
ConceptSubspace Locking

Subspace Locking

The phenomenon in model training whereby capabilities from different domains are encoded into relatively fixed and separated subspaces of the representation space; subspaces formed early are difficult to reallocate or share across domains, reducing the utilization of the representation space.

Timeline (last 90 days)

Nov 13

In terms of spatial dimensions, the researchers summarized two types of subspace locking phenomena: intra-domain and cross-domain.

Unverified50%
Nov 13

Subspace locking refers to the phenomenon in which the model encodes the capabilities of different domains into relatively fixed and separated subspaces in the representation space during training; once formed early on, it is difficult for subsequent training to reallocate or share them.

Unverified50%
Nov 13

The concept of subspace locking draws on research on feature subspaces and task interference in representation learning, providing a geometric perspective for understanding negative transfer in multi-task learning.

Unverified50%

All Facts (3)

Source Articles