Operator-Transformer

Code for reproducing "Transformer for Partial Differential Equations' Operator Learning" (paper).

(Updated Aug31/2024) OFormer code has been re-organized will become part of the neuraloperator library in the future release. Check it out!

Models prediction on 2D Incompressible flow

For instruction on different cases, please go the corresponding subfolder. These codes are tested under PyTorch 1.8.1 on Ubuntu 18.

Datasets for 1D Burgers/2D Darcy flow/2D Navier-Stokes (uniform equidist grid)

The dataset for 1D Burgers (Burgers_R10.zip), 2D Darcy flow (Darcy_421.zip) can be downloaded from dataset link .
We provide our processed dataset for 2D Navier-Stokes (in .npy format) at dataset link .
The dataset for these problems are under the courtesy of FNO.

Datasets for BVP problem on non-uniform grid

Dataset courtesy under GNN-BVP, please check the original repo for data downloading.

Datasets for IVP problem on Airfoil

Dataset courtesy under MeshGraphNet, we provide our processed dataset at train/test.

Pretrained model checkpoint and log

Problem	link
NS2D-Re200	link
NS2D-mixRe	link
NS2D-Re20	link
Burgers	link
Darcy	link
Airfoil	link
Electrostatics	link
Magnetostatics	link

Relevant works

Alongside the aforementioned projects that have generously shared their valuable datasets, the following repositories have also been helpful for this project.

• Galerkin Transformer: https://github.com/scaomath/galerkin-transformer
• A lot useful modules for building Transformer: https://github.com/lucidrains/x-transformers

Citations

If you find this project useful, please consider citing our work:

@article{
li2023transformer,
title={Transformer for Partial Differential Equations{\textquoteright} Operator Learning},
author={Zijie Li and Kazem Meidani and Amir Barati Farimani},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2023},
url={https://openreview.net/forum?id=EPPqt3uERT},
note={}
}