Intrinsic Dimensionality
A concept in neural network optimization research referring to the minimum number of degrees of freedom actually required to optimize a network's parameters. Research indicates that for many tasks, optimization within a random subspace of far lower dimensionality than the full parameter space can achieve performance comparable to full-parameter training.
Timeline (last 90 days)
Intrinsic dimension refers to the dimension of the low-dimensional manifold or subspace in which high-dimensional data points actually reside, which is usually much smaller than the ambient dimension.
For learning tasks operating on an intrinsic d*-dimensional submanifold, the paper proves that the optimal partition scaling can achieve the minimax nonparametric rate O(n^{-1/(d*+2)}√ln n).
All Facts (2)
Intrinsic dimension refers to the dimension of the low-dimensional manifold or subspace in which high-dimensional data points actually reside, which is usually much smaller than the ambient dimension.
50%UnverifiedFor learning tasks operating on an intrinsic d*-dimensional submanifold, the paper proves that the optimal partition scaling can achieve the minimax nonparametric rate O(n^{-1/(d*+2)}√ln n).
50%