Bi-Level Optimization
An optimization framework that partitions parameters into upper-level and lower-level groups, alternately optimizing each under different objectives or on different data. It is widely applied in scenarios such as meta-learning and neural architecture search.
Timeline (last 90 days)
In EDMD-kDL, the outer optimization learns the parameters of the kernel dictionary, and the inner optimization solves for a finite-dimensional approximation of the Koopman operator under that dictionary, with the two iterating alternately.
The general form of bilevel optimization is min_θ F(θ, φ*(θ)), where φ*(θ) = argmin_φ G(θ, φ), and its gradient can be computed efficiently via the implicit function theorem or iterative differentiation techniques.
The bilevel optimization decomposes the problem into an outer leader responsible for backbone network learning and an inner follower responsible for calibrator adjustment, with both converging cooperatively through a game within the same training loop.
The challenge of bilevel optimization lies in gradient propagation for the inner problem; computing outer-level gradients requires the implicit function theorem or approximate expansion, which generally incurs high computational cost.
All Facts (4)
In EDMD-kDL, the outer optimization learns the parameters of the kernel dictionary, and the inner optimization solves for a finite-dimensional approximation of the Koopman operator under that dictionary, with the two iterating alternately.
50%UnverifiedThe general form of bilevel optimization is min_θ F(θ, φ*(θ)), where φ*(θ) = argmin_φ G(θ, φ), and its gradient can be computed efficiently via the implicit function theorem or iterative differentiation techniques.
50%UnverifiedThe bilevel optimization decomposes the problem into an outer leader responsible for backbone network learning and an inner follower responsible for calibrator adjustment, with both converging cooperatively through a game within the same training loop.
50%UnverifiedThe challenge of bilevel optimization lies in gradient propagation for the inner problem; computing outer-level gradients requires the implicit function theorem or approximate expansion, which generally incurs high computational cost.
50%