Conceptintrinsic dimension / 固有维度

Intrinsic Dimensionality

A concept in neural network optimization research referring to the minimum number of degrees of freedom actually required to optimize a network's parameters. Research indicates that for many tasks, optimization within a random subspace of far lower dimensionality than the full parameter space can achieve performance comparable to full-parameter training.

Timeline (last 90 days)

Oct 16

For learning tasks operating on an intrinsic d*-dimensional submanifold, the paper proves that the optimal partition scaling can achieve the minimax nonparametric rate O(n^{-1/(d*+2)}√ln n).

Unverified50%

All Facts (1)

Unverified

For learning tasks operating on an intrinsic d*-dimensional submanifold, the paper proves that the optimal partition scaling can achieve the minimax nonparametric rate O(n^{-1/(d*+2)}√ln n).

50%

Source Articles

Understand LoRA in Five Minutes: Why rank-8 Fine-Tuning Is EnoughOct 16 MaRN: A PyTorch Library for Training Neural Networks via Low-Dimensional Parameter MappingsOct 9