[KongchangAI]
Concept

DiffPCA

A method for extracting a language subspace from model activations. It first computes the activation differences of the same semantics across different languages, then applies PCA to the difference matrix to capture language-transformation directions.

Timeline (last 90 days)

Nov 4

The study used two methods for constructing language subspaces: the existing Mean-SVD construction and DiffPCA, proposed by the authors.

Unverified50%
Nov 4

Both the Mean-SVD and DiffPCA subspaces captured a considerable degree of translation-related variation and exhibited similar patterns across different model depths.

Unverified50%
Nov 4

After projecting out the Mean-SVD or DiffPCA subspaces, linear probes could still almost perfectly identify the language, indicating that neither method exhaustively captures language information.

Unverified50%

All Facts (3)

Source Articles