Conceptmoment relaxation
Moment Relaxation
A convex relaxation technique in which auxiliary variables represent the original nonlinear terms and constraints such as positive semidefiniteness are imposed, thereby relaxing a non-convex problem into a computable convex upper-bound problem. The relaxation yields a tractable bound whose solution upper-bounds the optimum of the original problem.
Timeline (last 90 days)
Nov 3
On GPT-2, the moment relaxation certificate is tighter than linear bound propagation on every attention head.
Unverified50%