Out-of-Sample R²
Out-of-sample R² is a metric for the generalization ability of a predictive model on new samples. It is usually defined as one minus the ratio of the sum of squared prediction residuals to the total sum of squares computed against the sample mean as a baseline. The value can fall below zero.
Timeline (last 90 days)
The closed-form expression derived in the paper relates the expected out-of-sample R² and the expected correlation coefficient r to four quantities that can be specified at the design stage: the sample size N, the number of predictor variables p, the assumed population R² (ρ²_in), and the design efficiency factor D*.
Out-of-sample R² is defined as 1 - Σ(y_i - ŷ_i)² / Σ(y_i - ȳ)², and when the model's predictions are worse than simply using the mean, this value drops below 0.
The squared relationship between out-of-sample R² and the correlation coefficient r does not hold strictly, because the denominator of R² is fixed to the mean baseline, whereas r² varies with the variance of the predicted values.
All Facts (3)
The closed-form expression derived in the paper relates the expected out-of-sample R² and the expected correlation coefficient r to four quantities that can be specified at the design stage: the sample size N, the number of predictor variables p, the assumed population R² (ρ²_in), and the design efficiency factor D*.
50%UnverifiedOut-of-sample R² is defined as 1 - Σ(y_i - ŷ_i)² / Σ(y_i - ȳ)², and when the model's predictions are worse than simply using the mean, this value drops below 0.
50%UnverifiedThe squared relationship between out-of-sample R² and the correlation coefficient r does not hold strictly, because the denominator of R² is fixed to the mean baseline, whereas r² varies with the variance of the predicted values.
50%