PCGrad
A gradient balancing strategy in multi-task learning that performs projection to resolve conflicts when task gradient directions conflict, thereby alleviating gradient conflicts among tasks.
Timeline (last 90 days)
GradNorm dynamically adjusts the loss weights of each task to make gradient norms converge, and PCGrad projects and resolves conflicts when gradient directions clash.
Representative methods for gradient balancing include PCGrad (projecting conflicting gradients onto a conflict-free subspace) and MGDA (a multiple gradient descent algorithm that finds a direction allowing all objectives to descend simultaneously).
All Facts (2)
GradNorm dynamically adjusts the loss weights of each task to make gradient norms converge, and PCGrad projects and resolves conflicts when gradient directions clash.
50%UnverifiedRepresentative methods for gradient balancing include PCGrad (projecting conflicting gradients onto a conflict-free subspace) and MGDA (a multiple gradient descent algorithm that finds a direction allowing all objectives to descend simultaneously).
50%