Functional Gradient Descent

An optimization method that performs gradient descent directly in function space rather than over parameter vectors. Its theoretical foundations draw from reproducing kernel Hilbert spaces (RKHS) and variational methods; each iteration updates a function. Gradient Boosting can be viewed as a special case of this framework.