HotPOT
HotPOT is a heterogeneous mixture-of-experts model for PDE pretraining that adopts a neural-numerical expert architecture, combining learnable predictors with fixed numerical updates to exploit numerical-scheme priors and reduce cross-task interference.
Timeline (last 90 days)
HotPOT adopts an overall architecture of a shared encoder plus heterogeneous experts, with each PDE family corresponding to a neural-numerical expert, in which a learnable predictor produces the quantities required for the numerical update and a fixed numerical update computes the next time-step state from these quantities.
HotPOT was tested on five PDE families under six settings, covering 24 same-scale comparisons before and after fine-tuning, and outperformed MoE-POT in all comparisons, reducing rollout error by an average of 50.7%.
The 17M-parameter HotPOT model surpassed the significantly larger MoE-POT model.
HotPOT showed performance gains in zero-shot prediction under unseen physical conditions, indicating that it learned transferable numerical-physical representations rather than overfitting to the training equation families.
HotPOT's code has been open-sourced on GitHub at the repository cruiseresearchgroup/HotPOT-code.
The applicable boundaries of HotPOT include: the fixed numerical update requires that a clear numerical scheme be available for each PDE family; for problems where numerical schemes are not yet mature or the equation form is uncertain, the construction cost of heterogeneous experts increases; and generalization ability across more equation types still requires further validation.
HotPOT embeds the known numerical structure of the equation as non-trainable fixed components into experts, letting the learnable parts focus on quantities that numerical updates cannot directly provide; this is a pretraining improvement path different from stacking parameters and data.
All Facts (7)
HotPOT showed performance gains in zero-shot prediction under unseen physical conditions, indicating that it learned transferable numerical-physical representations rather than overfitting to the training equation families.
50%UnverifiedHotPOT was tested on five PDE families under six settings, covering 24 same-scale comparisons before and after fine-tuning, and outperformed MoE-POT in all comparisons, reducing rollout error by an average of 50.7%.
50%UnverifiedThe 17M-parameter HotPOT model surpassed the significantly larger MoE-POT model.
50%UnverifiedHotPOT adopts an overall architecture of a shared encoder plus heterogeneous experts, with each PDE family corresponding to a neural-numerical expert, in which a learnable predictor produces the quantities required for the numerical update and a fixed numerical update computes the next time-step state from these quantities.
50%UnverifiedHotPOT embeds the known numerical structure of the equation as non-trainable fixed components into experts, letting the learnable parts focus on quantities that numerical updates cannot directly provide; this is a pretraining improvement path different from stacking parameters and data.
50%UnverifiedThe applicable boundaries of HotPOT include: the fixed numerical update requires that a clear numerical scheme be available for each PDE family; for problems where numerical schemes are not yet mature or the equation form is uncertain, the construction cost of heterogeneous experts increases; and generalization ability across more equation types still requires further validation.
50%UnverifiedHotPOT's code has been open-sourced on GitHub at the repository cruiseresearchgroup/HotPOT-code.
50%