Gradient Checking
A debugging technique that approximates analytical gradients using numerical differentiation and compares them with backpropagation results to verify implementation correctness. It typically requires a relative error below 1e-7.
Timeline (last 90 days)
Gradient checking is a method that uses numerical differentiation to approximate the analytical gradient and compares it with the backpropagation result for relative error, usually requiring the relative error to be less than 10^-7.
In practice, it is recommended to first perform gradient checking on a very small network with a small number of samples, and then scale up to the full model after confirming there are no errors.
All Facts (2)
Gradient checking is a method that uses numerical differentiation to approximate the analytical gradient and compares it with the backpropagation result for relative error, usually requiring the relative error to be less than 10^-7.
50%UnverifiedIn practice, it is recommended to first perform gradient checking on a very small network with a small number of samples, and then scale up to the full model after confirming there are no errors.
50%