Staged Training
A two-step method for training closed-loop models: first, data assimilation is used to infer states and closed-loop trajectories; then a machine learning closed-loop model is trained offline, avoiding repeated calls to the simulator during training.
Timeline (last 90 days)
A recent study compared embedded training and staged training as two data-driven closed-loop modeling methods.
The contributions between stages cannot be simply decomposed, and the objectives of different stages may conflict, causing the model to lose old knowledge while acquiring new skills.
The paper proposes a staged training strategy: in the first stage, all layers are trained normally; in the second stage, the recurrent layer weights are frozen and only the attention layer and other parts are updated, to mitigate competitive learning.
Through ablation experiments, the paper shows that freezing the recurrent layer at a mid-stage is more effective than freezing throughout or freezing at a late stage.
In the future, the staged training idea can be extended to more complex hybrid architecture designs, such as dynamically adjusting the update pace of different layers or introducing more fine-grained attribution analysis to guide training strategies.
All Facts (5)
The contributions between stages cannot be simply decomposed, and the objectives of different stages may conflict, causing the model to lose old knowledge while acquiring new skills.
50%UnverifiedThe paper proposes a staged training strategy: in the first stage, all layers are trained normally; in the second stage, the recurrent layer weights are frozen and only the attention layer and other parts are updated, to mitigate competitive learning.
50%UnverifiedThrough ablation experiments, the paper shows that freezing the recurrent layer at a mid-stage is more effective than freezing throughout or freezing at a late stage.
50%UnverifiedA recent study compared embedded training and staged training as two data-driven closed-loop modeling methods.
50%UnverifiedIn the future, the staged training idea can be extended to more complex hybrid architecture designs, such as dynamically adjusting the update pace of different layers or introducing more fine-grained attribution analysis to guide training strategies.
50%