Mixture of Neural Operators: Incorporating Historical Information for Longer Rollouts

Abstract

Traditional numerical solvers for time-dependent partial differential equations (PDEs) notoriously require high computational resources and necessitate recomputation when faced with new problem parameters. In recent years, neural surrogates have shown great potential to overcome these limitations. However, it has been paradoxically observed that incorporating historical information into neural surrogates worsens their rollout performance. Drawing inspiration from multistep methods that use historical information from previous steps to obtain higher-order accuracy, we introduce the Mixture of Neural Operators (MoNO) framework; a collection of neural operators, each dedicated to processing information from a distinct previous step. We validate MoNO on the Kuramoto-Sivashinsky equation, demonstrating enhanced accuracy and stability of longer rollouts, greatly outperforming neural operators that discard historical information.

Please cite this paper as:

@inproceedings{majid2024mixture,
  title={Mixture of neural operators: Incorporating historical information for longer rollouts},
  author={Majid, Harris Abdul and Tudisco, Francesco},
  booktitle={ICLR 2024 Workshop on AI4DifferentialEquations In Science},
  year={2024}
}

Keywords: Deep learning neural networks AI4Science Neural Operators