Deterministic Equivalent
A deterministic expression tool used in high-dimensional statistics to approximate random risk. When the dimension and sample size tend to infinity at the same ratio, the random risk converges in probability to this deterministic expression.
Timeline (last 90 days)
The study establishes deterministic equivalents for the risk of the two-step procedure, covering both model shift and covariance shift.
The deterministic equivalent is an approximation tool used in high-dimensional statistics, where as the dimension and sample size grow proportionally to infinity, the difference between the random risk and the deterministic expression converges to zero in probability.
The paper uses random matrix theory to derive deterministic equivalents of the population error for an optimally trained teacher model and a student model trained via gradient flow
The most significant technical contribution of this paper is the derivation of a deterministic equivalent expression for test error.
Based on deterministic equivalents, hyperparameters (weights for each source and the ridge regression regularization coefficient) can be optimized, and it can be determined whether a private external dataset is useful without accessing the data.
The deterministic equivalent expression for test error uniformly characterizes the interplay among factors such as sample size, covariance structure, model drift, regularization strength, and privacy noise.
Deterministic equivalents are a core tool of random matrix theory: when the dimension p and sample size n grow to infinity proportionally (p/n → γ), random quantities converge in probability to deterministic expressions.
All Facts (7)
The study establishes deterministic equivalents for the risk of the two-step procedure, covering both model shift and covariance shift.
50%UnverifiedThe deterministic equivalent is an approximation tool used in high-dimensional statistics, where as the dimension and sample size grow proportionally to infinity, the difference between the random risk and the deterministic expression converges to zero in probability.
50%UnverifiedThe paper uses random matrix theory to derive deterministic equivalents of the population error for an optimally trained teacher model and a student model trained via gradient flow
50%UnverifiedThe most significant technical contribution of this paper is the derivation of a deterministic equivalent expression for test error.
50%UnverifiedBased on deterministic equivalents, hyperparameters (weights for each source and the ridge regression regularization coefficient) can be optimized, and it can be determined whether a private external dataset is useful without accessing the data.
50%UnverifiedThe deterministic equivalent expression for test error uniformly characterizes the interplay among factors such as sample size, covariance structure, model drift, regularization strength, and privacy noise.
50%UnverifiedDeterministic equivalents are a core tool of random matrix theory: when the dimension p and sample size n grow to infinity proportionally (p/n → γ), random quantities converge in probability to deterministic expressions.
50%