[KongchangAI]
Conceptgradient checking / 数值梯度检验

Gradient Checking

A debugging technique that approximates analytical gradients using numerical differentiation and compares them with backpropagation results to verify implementation correctness. It typically requires a relative error below 1e-7.

Timeline (last 90 days)

Nov 6

Gradient checking is a method that uses numerical differentiation to approximate the analytical gradient and compares it with the backpropagation result for relative error, usually requiring the relative error to be less than 10^-7.

Unverified50%
Nov 6

In practice, it is recommended to first perform gradient checking on a very small network with a small number of samples, and then scale up to the full model after confirming there are no errors.

Unverified50%

All Facts (2)

Source Articles