Ordinary Differential Equation
A differential equation containing one independent variable and its derivatives, used to describe continuous change processes. In deep learning theory it is often used to analyze gradient descent in a continuous formulation.
Timeline (last 90 days)
This theoretical study uses the teacher-student model to derive, in the limit as the feature dimension D tends to infinity, ordinary differential equations that precisely describe the learning dynamics of the student network.
In the teacher-student model, the continuous-time limit of online stochastic gradient descent (online SGD) as the learning rate tends to zero is described by a set of ordinary differential equations.
All Facts (2)
This theoretical study uses the teacher-student model to derive, in the limit as the feature dimension D tends to infinity, ordinary differential equations that precisely describe the learning dynamics of the student network.
50%UnverifiedIn the teacher-student model, the continuous-time limit of online stochastic gradient descent (online SGD) as the learning rate tends to zero is described by a set of ordinary differential equations.
50%