add Extformer-MoE example by HKUST(GZ) by KennyNH · Pull Request #933 · PaddlePaddle/PaddleScience

KennyNH · 2024-06-26T07:25:30Z

We add a new example named Extformer-MoE. All the related codes are committed to this "dev_model" branch. Attached is a Markdown doc which includes some detailed descriptions.
开发文档.md

paddle-bot · 2024-06-26T07:25:34Z

Thanks for your contribution!

CLAassistant · 2024-06-26T07:25:35Z

All committers have signed the CLA.

HydrogenSulfate

辛苦提交PR，没有大的问题，几处小的问题麻烦再看一下谢谢

Co-authored-by: HydrogenSulfate <490868991@qq.com>

HydrogenSulfate · 2024-06-28T10:37:06Z

@KennyNH 可以在PaddleScience目录下执行 pre-commit run --all-files，否则code-style-check会失败

KennyNH · 2024-06-29T02:50:03Z

你好，已经重新 request a review

HydrogenSulfate · 2024-06-29T10:00:02Z

你好，已经重新 request a review

code-style-check还是挂了，可以确认一下pre-commit在commit时正常启用了，

以及，可以点击code-style-check的Details，查看具体未通过pre-commit的文件，

HydrogenSulfate

@KennyNH 麻烦用pre-commit重新格式化一下代码，通过code-style测试

KennyNH · 2024-07-01T09:28:39Z

已重新提交

HydrogenSulfate · 2024-07-01T09:48:36Z

已重新提交

好像还是挂了，可以确认一下git commit的时候，pre-commit插件被正确启用，并且所有检查都没有出现失败的情况

KennyNH · 2024-07-01T13:21:35Z

现在好像可以了

KennyNH · 2024-07-01T13:22:58Z

请问 PaddleScience-Linux-CI 这个是什么问题？

HydrogenSulfate · 2024-07-03T03:50:21Z

请问 PaddleScience-Linux-CI 这个是什么问题？

CI问题已经解决，但是我本地测试的时候发现运行会报错，是需要安装xarray的某一个特定后端吗？
已按照文档修改 FILE_PATH，并且打开的.nc文件也是存在的

Error executing job with overrides: []
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info
    file = self._cache[self._key]
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/lru_cache.py", line 56, in __getitem__
    value = self._cache[key]
KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/enso_round1_train_20210201/CMIP_train.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), 'fca25123-e359-4418-a25c-5a9da11389f8']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/extformer_moe_enso_train.py", line 195, in main
    train(cfg)
  File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/extformer_moe_enso_train.py", line 50, in train
    sup_constraint = ppsci.constraint.SupervisedConstraint(
  File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/constraint/supervised_constraint.py", line 64, in __init__
    _dataset = dataset.build_dataset(dataloader_cfg["dataset"])
  File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/__init__.py", line 100, in build_dataset
    dataset = eval(dataset_cls)(**cfg)
  File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/ext_moe_enso_dataset.py", line 292, in __init__
    cmip6sst, cmip5sst, cmip6nino, cmip5nino = read_raw_data(self.data_dir)
  File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/ext_moe_enso_dataset.py", line 142, in read_raw_data
    train_cmip = xr.open_dataset(Path(ds_dir) / "CMIP_train.nc").transpose(
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/api.py", line 573, in open_dataset
    backend_ds = backend.open_dataset(
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 646, in open_dataset
    store = NetCDF4DataStore.open(
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 409, in open
    return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 356, in __init__
    self.format = self.ds.data_model
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 418, in ds
    return self._acquire()
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 412, in _acquire
    with self._manager.acquire_context(needs_lock) as root:
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 199, in acquire_context
    file, cached = self._acquire_with_cache_info(needs_lock)
  File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info
    file = self._opener(*self._args, **kwargs)
  File "src/netCDF4/_netCDF4.pyx", line 2470, in netCDF4._netCDF4.Dataset.__init__
  File "src/netCDF4/_netCDF4.pyx", line 2107, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -101] NetCDF: HDF error: '/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/enso_round1_train_20210201/CMIP_train.nc'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

KennyNH · 2024-07-03T04:09:15Z

enso 数据集路径和 cfg 文件中路径相符吗？如果不是这个问题我稍后 check一下获取Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: HydrogenSulfate ***@***.***> Sent: Wednesday, July 3, 2024 11:50:44 AM To: PaddlePaddle/PaddleScience ***@***.***> Cc: Hang NI ***@***.***>; Mention ***@***.***> Subject: Re: [PaddlePaddle/PaddleScience] add Extformer-MoE example by HKUST(GZ) (PR #933) 请问 PaddleScience-Linux-CI 这个是什么问题？ CI问题已经解决，但是我本地测试的时候发现运行会报错，是需要安装xarray的某一个特定后端吗？ Error executing job with overrides: [] Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info file = self._cache[self._key] File "/usr/local/lib/python3.10/dist-packages/xarray/backends/lru_cache.py", line 56, in __getitem__ value = self._cache[key] KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/enso_round1_train_20210201/CMIP_train.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), 'fca25123-e359-4418-a25c-5a9da11389f8'] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/extformer_moe_enso_train.py", line 195, in main train(cfg) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/extformer_moe_enso_train.py", line 50, in train sup_constraint = ppsci.constraint.SupervisedConstraint( File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/constraint/supervised_constraint.py", line 64, in __init__ _dataset = dataset.build_dataset(dataloader_cfg["dataset"]) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/__init__.py", line 100, in build_dataset dataset = eval(dataset_cls)(**cfg) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/ext_moe_enso_dataset.py", line 292, in __init__ cmip6sst, cmip5sst, cmip6nino, cmip5nino = read_raw_data(self.data_dir) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/ext_moe_enso_dataset.py", line 142, in read_raw_data train_cmip = xr.open_dataset(Path(ds_dir) / "CMIP_train.nc").transpose( File "/usr/local/lib/python3.10/dist-packages/xarray/backends/api.py", line 573, in open_dataset backend_ds = backend.open_dataset( File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 646, in open_dataset store = NetCDF4DataStore.open( File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 409, in open return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose) File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 356, in __init__ self.format = self.ds.data_model File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 418, in ds return self._acquire() File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4_.py", line 412, in _acquire with self._manager.acquire_context(needs_lock) as root: File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 199, in acquire_context file, cached = self._acquire_with_cache_info(needs_lock) File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info file = self._opener(*self._args, **kwargs) File "src/netCDF4/_netCDF4.pyx", line 2470, in netCDF4._netCDF4.Dataset.__init__ File "src/netCDF4/_netCDF4.pyx", line 2107, in netCDF4._netCDF4._ensure_nc_success OSError: [Errno -101] NetCDF: HDF error: '/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/enso_round1_train_20210201/CMIP_train.nc' Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace. ― Reply to this email directly, view it on GitHub<#933 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/APBKFG6DAT2KT2BACNQ7NOTZKNYJJAVCNFSM6AAAAABJ5IYE6CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBVGAZTKMBUGY>. You are receiving this because you were mentioned.Message ID: ***@***.***>

HydrogenSulfate · 2024-07-03T05:13:43Z

enso 数据集路径和 cfg 文件中路径相符吗？如果不是这个问题我稍后 check一下获取Outlook for Androidhttps://aka.ms/AAb9ysg
…
________________________________ From: HydrogenSulfate @.> Sent: Wednesday, July 3, 2024 11:50:44 AM To: PaddlePaddle/PaddleScience @.> Cc: Hang NI @.>; Mention @.> Subject: Re: [PaddlePaddle/PaddleScience] add Extformer-MoE example by HKUST(GZ) (PR #933) 请问 PaddleScience-Linux-CI 这个是什么问题？ CI问题已经解决，但是我本地测试的时候发现运行会报错，是需要安装xarray的某一个特定后端吗？ Error executing job with overrides: [] Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info file = self._cache[self.key] File "/usr/local/lib/python3.10/dist-packages/xarray/backends/lru_cache.py", line 56, in getitem value = self.cache[key] KeyError: [<class 'netCDF4.netCDF4.Dataset'>, ('/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/enso_round1_train_20210201/CMIP_train.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False)), 'fca25123-e359-4418-a25c-5a9da11389f8'] During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/extformer_moe_enso_train.py", line 195, in main train(cfg) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/extformer_moe_enso_train.py", line 50, in train sup_constraint = ppsci.constraint.SupervisedConstraint( File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/constraint/supervised_constraint.py", line 64, in init dataset = dataset.build_dataset(dataloader_cfg["dataset"]) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/init.py", line 100, in build_dataset dataset = eval(dataset_cls)(**cfg) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/ext_moe_enso_dataset.py", line 292, in init cmip6sst, cmip5sst, cmip6nino, cmip5nino = read_raw_data(self.data_dir) File "/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/ppsci/data/dataset/ext_moe_enso_dataset.py", line 142, in read_raw_data train_cmip = xr.open_dataset(Path(ds_dir) / "CMIP_train.nc").transpose( File "/usr/local/lib/python3.10/dist-packages/xarray/backends/api.py", line 573, in open_dataset backend_ds = backend.open_dataset( File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4.py", line 646, in open_dataset store = NetCDF4DataStore.open( File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4.py", line 409, in open return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose) File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4.py", line 356, in init self.format = self.ds.data_model File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4.py", line 418, in ds return self.acquire() File "/usr/local/lib/python3.10/dist-packages/xarray/backends/netCDF4.py", line 412, in _acquire with self._manager.acquire_context(needs_lock) as root: File "/usr/lib/python3.10/contextlib.py", line 135, in enter return next(self.gen) File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 199, in acquire_context file, cached = self._acquire_with_cache_info(needs_lock) File "/usr/local/lib/python3.10/dist-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info file = self._opener(self._args, kwargs) File "src/netCDF4/_netCDF4.pyx", line 2470, in netCDF4._netCDF4.Dataset.init File "src/netCDF4/_netCDF4.pyx", line 2107, in netCDF4._netCDF4._ensure_nc_success OSError: [Errno -101] NetCDF: HDF error: '/ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/enso_round1_train_20210201/CMIP_train.nc' Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace. ― Reply to this email directly, view it on GitHub<#933 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APBKFG6DAT2KT2BACNQ7NOTZKNYJJAVCNFSM6AAAAABJ5IYE6CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBVGAZTKMBUGY. You are receiving this because you were mentioned.Message ID: @.>

是的，路径是：FILE_PATH: /ssd2/sjx/sjx_cuda11.8_py310/PaddleScience_test/PaddleScience/examples/extformer_moe/enso_round1_train_20210201

KennyNH · 2024-07-06T08:36:37Z

你好！我这边好像没发现有类似问题。xarray 的 pip show 信息如下，有没有可能是版本问题呢？

Name: xarray
Version: 2024.2.0
Summary: N-D labeled arrays and datasets in Python
Home-page:
Author:
Author-email: xarray Developers xarray@googlegroups.com
License: Apache-2.0
Location: /hpc2hdd/home/hni017/.conda/envs/mt/lib/python3.9/site-packages
Requires: numpy, packaging, pandas
Required-by: earthnet

HydrogenSulfate

几处地方可以一起改一下，另外想问一下这个模型需要多大的显存呢，16G的似乎不够？

KennyNH · 2024-07-08T02:36:04Z

几处地方可以一起改一下，另外想问一下这个模型需要多大的显存呢，16G的似乎不够？

大概30-40G 左右，如果显存不够可以设置 checkpoint_level = 3

HydrogenSulfate · 2024-07-08T02:36:46Z

几处地方可以一起改一下，另外想问一下这个模型需要多大的显存呢，16G的似乎不够？

大概30-40G 左右，如果显存不够可以设置 checkpoint_level = 3

好的谢谢，我用A100跑训练看看

Co-authored-by: HydrogenSulfate <490868991@qq.com>

HydrogenSulfate

LGTM

* add extformer-moe * fix 0624 * fix 0624 * Delete outputs_extformer_moe_pretrain directory * Delete docs/zh/examples/extformer_moe_figs directory * Update extformer_moe.md * Update extformer_moe_utils.py * Update extformer_moe.md * Update ppsci/data/dataset/__init__.py Co-authored-by: HydrogenSulfate <490868991@qq.com> * Update ppsci/data/dataset/ext_moe_enso_dataset.py Co-authored-by: HydrogenSulfate <490868991@qq.com> * fix merge review issues * fix merge 240629 * fix 240701 * fix 240701_ * fix 240701__ * fix 240701___ * Update ext_moe_enso_dataset.py * Update docs/zh/examples/extformer_moe.md Co-authored-by: HydrogenSulfate <490868991@qq.com> * Update docs/zh/examples/extformer_moe.md Co-authored-by: HydrogenSulfate <490868991@qq.com> * Update ppsci/data/dataset/ext_moe_enso_dataset.py --------- Co-authored-by: HydrogenSulfate <490868991@qq.com>

luotao1 · 2024-08-13T06:36:19Z

hi, @KennyNH

非常感谢你对飞桨的贡献，我们正在运营一个PFCC组织，会通过定期分享技术知识与发布开发者主导任务的形式持续为飞桨做贡献，详情可见 https://github.com/luotao1 主页说明。
如果你对PFCC有兴趣，请发送邮件至 ext_paddle_oss@baidu.com，我们会邀请你加入~

…addlePaddle#967) * add Solver.finetune() (PaddlePaddle#755) * 【PPSCI Doc No.20】ppsci.arch.DeepPhyLSTM (PaddlePaddle#756) * [Add] phylstm examples * [Change] phylstm examples * [Change] remove blank line * clear differentiation cache in visu_forward to avoid OOM problem (PaddlePaddle#757) * update docstring (PaddlePaddle#758) * 【PPSCI Doc No.104-105】 (PaddlePaddle#759) * update docstring * remove spaces in args * update * remove extra blank line * 【Hackathon 5th No.63】 PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs (PaddlePaddle#674) * Add files via upload * Create readme.md * Add files via upload * Update __init__.py * Add files via upload * Add files via upload * add * add * add * add * add * add * add * add * add * add * a * a * Update phycrnet.md * Update phycrnet.md * Update phycrnet.md * a * d * d * a * a * a * a * a * a * p * a * a * a * update .md file * update .md file * update .md file * fix * edit md * eval * m * m * edit * edit --------- Co-authored-by: WG <39621324+wangguan1995@users.noreply.github.com> Co-authored-by: wangguan <772359200@qq.com> * [Doc] Add more contrib and more installtion method (PaddlePaddle#760) * add 3 install method in README.md * add more contributors * [Doc] Add tutorial and fix docs (PaddlePaddle#766) * (WIP)Add tutorial page * fix document * fix * change split_to_dict and concat_to_tensor to staticmethod * update batch_index class variable for all datasets * move legend to outter-right of curve box and set dpi to 200 * add 1 more aistudio url * [Fix] Update copyright and docstrings (PaddlePaddle#761) * update copyright header * update docstring of PhyCRNet * doc106 (PaddlePaddle#762) * set encoding to utf-8 when reading content from README.md during pip install (PaddlePaddle#768) * doc108-109 (PaddlePaddle#764) * remove python3.7 in docker (PaddlePaddle#765) * remove python3.7 * update * 【PPSCI Doc No.107】update doc for `ppsci.visualize.save_vtu_to_mesh` (PaddlePaddle#763) * doc107 * fix * [Example] Add RegAE example (PaddlePaddle#660) * add RegAE example * add RegAE --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * [Enh] add validation for hydra config (PaddlePaddle#769) * add validation for hydra config * update unitest for pydantic * fix for OptimizerList * fix * 【Hackathon 5th No.54】NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations (PaddlePaddle#670) * Add files via upload * add * add * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * a * edit * m * edit * a * Update nsfnet4.md * m * m * Update nsfnet4.md * finish * m * debug * edit * add * a * a * a * a * a * a * a * d * Update nsfnet4.md * d * d * d * d * over * reformat docs * fix inference * fix lr scheduler --------- Co-authored-by: WG <39621324+wangguan1995@users.noreply.github.com> Co-authored-by: wangguan <772359200@qq.com> * specify pydantic >= 2.5.x or field_validator can not be imported from pydantic 1.x version (PaddlePaddle#770) * add YingLong model (PaddlePaddle#771) * add yinglong inference code * add readme file for yinglong * update yinglong predict code * add copyright in predict_12layers.py * update readme * add reference for timefeatures.py * update predict_12layers and readme * [Doc] Add description of yaml (PaddlePaddle#774) * (WIP)fix doctest and refine document * update docs * fix wrong code in ldc2d * update code * update paranoma * fix * refine doc * Yinglong infer (PaddlePaddle#772) * add yinglong inference code * add readme file for yinglong * update yinglong predict code * add copyright in predict_12layers.py * update readme * add reference for timefeatures.py * update predict_12layers and readme * add visualize and update readme * add figure for yinglong * update yinglong readme * fix readme for yinglong * add predict for yinglong 24 layers * update readme * add plot for yinglong * update predict code * update readme * update file path * add version check * fix doc string * update readme * update readme (PaddlePaddle#775) * set create_graph=False before eval (PaddlePaddle#776) * [Fea] Support python inference (PaddlePaddle#773) * [Doc] Add pretrained model for laplace2d & refine comments (PaddlePaddle#639) * update laplace2d pretrained model * remove 'after finished training' comment in evaluate function * update README.md * add deploy module for aneurysm * update code * update aneurysm code * update code * update code * update code * update aneurysm document * update export and inference document * fix docstring * [Fix] restore 'by_epoch' for SchedulerList and fix EPNN (PaddlePaddle#777) * restore 'by_epoch' for SchedulerList * fix for epnn * [Fix]fix data transform error (PaddlePaddle#779) * [Fix]fix data transform error * [Update]add 'auto_collation' for some examples * API 文档补全LorenzEmbedding,RosslerEmbedding * Update embedding_koopman.py * Update embedding_koopman.py * Revert "[Fix]fix data transform error (PaddlePaddle#779)" (PaddlePaddle#781) This reverts commit 6823a3e. * [Update]update setting of 'auto_collation' and fix errors (PaddlePaddle#783) * set 'auto_collation' false when using data transform * fix data transform error of topopt example * update document (PaddlePaddle#782) * update Graphcast to document (PaddlePaddle#784) * fix url of GraphCast in index.md (PaddlePaddle#785) * [Refine] Refine yinglong code (PaddlePaddle#786) * refine yinglong predictor and use config instead of argparser * modify README of yinglong * fix for review * add requirements for yinglong (PaddlePaddle#787) * [Docker] Update dockerfile & refine aneurysm (PaddlePaddle#789) * update pymesh package and dockerfile for python3.9 * refine aneurysm code for export and inference * install requirements.txt before executing setup.py for pymesh * revert random permutation from numpy to paddle (PaddlePaddle#792) * 【PPSCI Doc No.9】 ppsci.data.dataset.VtuDataset (PaddlePaddle#791) * Add docx * Apply suggestions from code review * Apply suggestions from code review * [Fea] Support onnx and TensorRT inference (PaddlePaddle#794) * support onnx inference in base Predictor and PINNPredictor * support exporting onnx model after exporting paddle inference models via argument 'with_onnx' * support TensorRT for aneurysm and add TensorRT example in document * 【PPSCI Export&Infer No. 29】 add export and inference (PaddlePaddle#793) * add export and inference * update * 【PPSCI Export&Infer No.3】euler_beam (PaddlePaddle#796) * 【PPSCI Export&Infer No.3】euler_beam * Update examples/euler_beam/euler_beam.py Co-authored-by: HydrogenSulfate <490868991@qq.com> * update config --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * Move 2 tensor initialization to lazy call to prevent using GPU memory when start program in non-gpu mode(such as cpu) (PaddlePaddle#795) * 【PPSCI Export&Infer No.4】laplace2d (PaddlePaddle#797) * [Doc] Fix pipe document & refine inference guide (PaddlePaddle#798) * correct pretrainde_model_path of Pipe Flow(test=document_fix) * refine inference document * fix significant figures to 2 for tidy plot * increase indention * [Fea] Add ModifiedMLP (PaddlePaddle#799) * Add ModifiedMLP * limit hidden_size to type int for ModifiedMLP * fix f-string * 【PPSCI Export&Infer No.5】lorenz (PaddlePaddle#801) * 【PPSCI Export&Infer No.5】lorenz * fix bug * 【PPSCI Export&Infer No.6】rossler (PaddlePaddle#803) * API 文档补全Generator (PaddlePaddle#804) * fix lorenz/rossler export and infer (PaddlePaddle#805) * add embedding model to PhysformerGPT2 for infer * modify export and inference code of lorenz * fix export command * fix export and infer of rossler * fix doc * fix error message in generate method * fix docstring * refine code (PaddlePaddle#806) * 【PPSCI Export&Infer No.7】 volterra_ide (PaddlePaddle#807) * 【PPSCI Export&Infer No.7】 volterra_ide * fix docstyle * Allow initialize dataloader without specifying 'sampler' (PaddlePaddle#809) * [Add]MRMSDataset (PaddlePaddle#810) * [Add]MRMSDataset * fix code and style * [Fea] Support tensorboardX and add corresponding guidance (PaddlePaddle#812) * support tensorboardX for viv as demo and add tensorboardX guide in user_guide.md * fix comma * Fix sot sci error (PaddlePaddle#815) * add nsys file * add nsys file * add nsys file * fix paddle sot error * fix code style * chip heat simulation (PaddlePaddle#808) * add bubble datafile test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code * add bubble data * add bubble code * delete mat file * delete mat file * bubble code * delete mat file * add some modify * add some modifications * add some modifications * add some modifications * add some modifications * add some modification * add some modification * add some modification * add some modification * add some modification * add some modification(test=document_fix) * add some modification * add some modification * add some modification * add some modification * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code for chip heat simulation * add some code for chip heat simulation * add some code for DGMR * add some code for chip * add some code for chip * deleted dgmr code * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * Update examples/chip_heat/conf/chip_heat.yaml --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * modified: docs/zh/examples/chip_heat.md (PaddlePaddle#820) * add bubble datafile test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code * add bubble data * add bubble code * delete mat file * delete mat file * bubble code * delete mat file * add some modify * add some modifications * add some modifications * add some modifications * add some modifications * add some modification * add some modification * add some modification * add some modification * add some modification * add some modification(test=document_fix) * add some modification * add some modification * add some modification * add some modification * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code for chip heat simulation * add some code for chip heat simulation * add some code for DGMR * add some code for chip * add some code for chip * deleted dgmr code * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * modified some code for chip heat * 【PPSCI Doc No.58、59、60】Mesh.from_pymesh、translate、scale (PaddlePaddle#818) * update mesh.py * update mesh.py * [Doc] Fix url in chip_heat document (PaddlePaddle#822) * add bubble datafile test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code * add bubble data * add bubble code * delete mat file * delete mat file * bubble code * delete mat file * add some modify * add some modifications * add some modifications * add some modifications * add some modifications * add some modification * add some modification * add some modification * add some modification * add some modification * add some modification(test=document_fix) * add some modification * add some modification * add some modification * add some modification * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code for chip heat simulation * add some code for chip heat simulation * add some code for DGMR * add some code for chip * add some code for chip * deleted dgmr code * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * modified some code for chip heat * modified some code for chip heat * [Fix] Fix doctest (PaddlePaddle#821) * fix(test=document_fix) * pretty evaluation output with prettytable * fix code failed in doctest * Revert "Merge branch 'pretty_eval_output' into develop" This reverts commit 66cded1, reversing changes made to 18ea97a. * skip mesh example code in geometry/mesh.py for requiring pymesh installtion * skip more code for doctest * [Fix]dtype of RadarDataset's data (PaddlePaddle#823) * 【PPSCI Doc No.38-40】 (PaddlePaddle#826) * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix --------- Co-authored-by: krp <2934631798@qq.com> * remove seed and directory code to slim laplace2d exmaples (PaddlePaddle#824) * [Fea] Support nvtx profiling (PaddlePaddle#825) * support nvtx profiling via NVTX=1 * rename trainer to solver * add user guide for NVTX * refine code of paddle.framework.core and add cache for 3 context_manager of Solver * update nsys chapter in user_guide * fix solver.py * Dgmr (PaddlePaddle#813) * add bubble datafile test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code * add bubble data * add bubble code * add some code for chip * add some code for DGMR * add some code for DGMR * add some code for DGMR * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * check if is directory befoe mkdirs (PaddlePaddle#827) * [Fix] Replace os.path.isdir with len (PaddlePaddle#830) * replace os.path.isdir with len * enumerate from 0 * add quasi-random sampling method: Halton (PaddlePaddle#828) * Update solver.py (PaddlePaddle#831) 修改osp.isdir为len * 【PPSCI Doc No.66-74】 (PaddlePaddle#829) * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * fix api docs in the timedomain * fix api docs of timedomain * fix api docs of timedomain --------- Co-authored-by: krp <2934631798@qq.com> Co-authored-by: HydrogenSulfate <490868991@qq.com> * [Enh] Refactor sum aggregator (PaddlePaddle#834) * add Sum loss aggregator * simplify loss aggregation code in train.py and add check for AGDA and PCGrad when used with amp * add check for using L-BFGS with use_amp=True * Refine Relobralo * Fix docstring of timedomain.py * remove unnecessary code in train.py * automatically download *.pdeqn file if available when download pretrained model * wrap func generated by symbolic module with DDP * fix Relobralo * initialize loss with 0.0 instead of first loss * 【PPSCI Doc No.41-57】 (PaddlePaddle#833) * fix docs bugs * fix docs bugs * fix codestyle bug * fix codestyle bugs * fix codestyle bugs * fix some bugs * fix codetype error * fix other bugs * fix typehint bugs * fix other bugs * fix codestyle bug * fix codestyle bug * add skip_prune_program arg for Solver.export (PaddlePaddle#835) * 【PPSCI Export&Infer No.23】viv (PaddlePaddle#832) * eadd export and inference for viv * add doc * fix viv export&infer * Rewriting function * fix viv export&infer * [Doc] Add more contributors and refine several docstrings (PaddlePaddle#836) * auto generate contributors instead of adding manually * refine return type of docstrings * 【PPSCI Doc No.61-65】 (PaddlePaddle#839) * fix doc bugs * fix codestyle bugs * [Doc] Fix description for mkldnn (PaddlePaddle#837) * fix description for mkldnn * merge 3 visualization chapter into one chapter * initialize loss with first constraint loss instead of 0.0, and change random seed of euler_beam for better l2error * update euler_beam doc * fix return type of PDE.set_state_dict * Update XPINN_2D_PoissonsEqn.py (PaddlePaddle#842) * support enabling prim via ++prim=1 (PaddlePaddle#843) * 【PPSCI Doc No.23、25-29、31-34、97】 (PaddlePaddle#840) * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * fix api docs in the timedomain * fix api docs of timedomain * fix api docs of timedomain * ppsci api docs fixed * ppsci api docs fixed * ppsci api docs fixed --------- Co-authored-by: krp <2934631798@qq.com> Co-authored-by: HydrogenSulfate <490868991@qq.com> * 【PPSCI Doc No.35-37】 (PaddlePaddle#846) * docs: add examples for api No.35-37 * Update ppsci/equation/pde/base.py * Update ppsci/equation/pde/base.py * Update ppsci/equation/pde/base.py --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * Bubble net (PaddlePaddle#847) * add bubble datafile test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code * add bubble data * add bubble code * delete mat file * delete mat file * bubble code * delete mat file * add some modify * add some modifications * add some modifications * add some modifications * add some modifications * add some modification * add some modification * add some modification * add some modification * add some modification * add some modification(test=document_fix) * add some modification * add some modification * add some modification * add some modification * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code test=develop * add some code for chip heat simulation * add some code for chip heat simulation * add some code for DGMR * add some code for chip * add some code for chip * deleted dgmr code * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * add some code for chip heat * modified some code for chip heat * modified some code for chip heat * modified some code for chip heat * [Example] Add allen cahn example (PaddlePaddle#845) * allow empty optimizer when saving checkpoint * add model averaging module * fix return dtype inconsistency with global dtype * use python func instead of sympy function for pow(u,3) get a bit poor L2 error than multiply(u*u*u) * refine AllenCahn docstring * support save and load for average model module * add 3 ema unitests * update 2023 to 2024 * add ema config pydantic scheme * add avg_range for SWA * update field_validator for swa and ema * support period embedding for MLP * Keep non-float data when reading file * update ema and save_load, printer and eval, solver module code * add allen_cahn example * refine code * save buffer and non-grad required params in ema * add unitest for ema with buffer * fix epoch_ema saving * add unitest for ema state_dict * refine allen_cahn_plain.py * fix string to floating conversion in reader.py * fix string to floating conversion in reader.py * remove print code in solver * Update allen_cahn_plain.py * Update misc.py --------- Co-authored-by: zzm <95690929+zhiminzhang0830@users.noreply.github.com> * 【benchmark】fix benchmark model name (PaddlePaddle#850) * add max_mem_reserved for benchmark * add max_mem_reserved for benchamrk * fix benchmark model name * [Example] Add allen_cahn causal train with fourier feature and random weight factorization (PaddlePaddle#848) * allow empty optimizer when saving checkpoint * add model averaging module * fix return dtype inconsistency with global dtype * use python func instead of sympy function for pow(u,3) get a bit poor L2 error than multiply(u*u*u) * refine AllenCahn docstring * support save and load for average model module * add 3 ema unitests * update 2023 to 2024 * add ema config pydantic scheme * add avg_range for SWA * update field_validator for swa and ema * support period embedding for MLP * Keep non-float data when reading file * update ema and save_load, printer and eval, solver module code * add allen_cahn example * refine code * save buffer and non-grad required params in ema * add unitest for ema with buffer * fix epoch_ema saving * add unitest for ema state_dict * refine allen_cahn_plain.py * fix string to floating conversion in reader.py * fix string to floating conversion in reader.py * update code and refine document * correct initialization for RWF * update docstring for arg 'random_weight' of mlp * update docstrings * add causal fourier rwf config * fix code in mlp.py * refine code in mse.py * add ema document file(test=document_fix) (PaddlePaddle#853) * [New example] Add nls-mb example (PaddlePaddle#838) * add NLS-MB example * fix * fix * fix * fix * modify * modify * modify * fix * replace deprecated 'FieldValidationInfo' with 'ValidationInfo' (PaddlePaddle#855) * check return type of FunctionalLoss (PaddlePaddle#854) * [Docker] Add docker image (PaddlePaddle#856) * execute 'ldconfig' after /bin/bash automatically * add_dockerhub * refine doc * update base image to paddle official image(test=document_fix) * update code * update code * 【PPSCI Export&Infer No.24】 biharmonic2d (PaddlePaddle#858) * [SCI Export&Infer No.24] biharmonic2d * P[PSCI Export&Infer No.724] biharmonic2d fix * [Fix] Fix NLS-MB document and code (PaddlePaddle#859) * execute 'ldconfig' after /bin/bash automatically * add_dockerhub * refine doc * update base image to paddle official image(test=document_fix) * Fix NLS-MB examples code and document * Update YingLong1 README.md (PaddlePaddle#861) * Update YingLong1 README.md * Update README.md * add '_' to initializer.glorot_normal * add allen_cahn_default.yaml * [Doc] Add anno for eval (PaddlePaddle#863) * add annotation for eval.py * add NLS example into index and README * rename yinglong to yinglong1 (PaddlePaddle#862) * add allen-cahn document * update pymesh install package * 【PPSCI Export&Infer No.22】VP_NSFNet4 (PaddlePaddle#864) * [SCI Export&Infer No.24] biharmonic2d * P[PSCI Export&Infer No.724] biharmonic2d fix * add export&infer nsfnet4 * add export&infer nsfnet4 * update pymesh install package (PaddlePaddle#865) * [Refine] Refine evaluation output (PaddlePaddle#866) * fix(test=document_fix) * pretty evaluation output with prettytable * remove epoch id when ony evaluating * hidden epoch information when not evaluating on flying * [Enh] Add numpy solver in config solvers (PaddlePaddle#869) * support registering numpy config solvers when parsing attribute of numpy from config * remove permutation code * avoid duplicated register * [API] Add chamfer loss (PaddlePaddle#871) * add chamfer loss * update development.md * update example of chamfer loss docstring * fix doctest for ChamferLoss * support batch comupute for chamferloss * add-earthformer (PaddlePaddle#870) * del arch.md in data (PaddlePaddle#872) * [Add]Add competition&IJCAI_2024 submodule (PaddlePaddle#873) * [Add]Add competition&IJCAI_2024 code * update README * add IJCAI 2024 CAR competition to inedx.md and fix docstrings in cuboid_transformer.py(test=document_fix) (PaddlePaddle#874) * [Update]update competition description (PaddlePaddle#876) * Fix typo (PaddlePaddle#881) * fix typo for correcting 'direcotry' to 'directory' * fix typo for correcting 'unchaned' to 'unchanged' * [Fix] Fix inference path typo (PaddlePaddle#880) * fix(test=document_fix) * fix typo for correcting 'pdpiparmas' to 'pdiparams' * add left bracket for docstring of CuboidSelfAttentionLayer * 【PPSCI Export&Infer No.15-16】 (PaddlePaddle#875) * fix doc bugs * fix codestyle bugs * 【PPSCI Export&Infer No.15-16】 * fix codestyle bug for PPSCI Export&Infer No.15-16】 * fix codestyle bugs for 【PPSCI Export&Infer No.15-16】 * fix codestyle bugs for 【PPSCI Export&Infer No.15-16】 * fix codestyle bugs for 【PPSCI Export&Infer No.15-16】 * fix bugs for 【PPSCI Export&Infer No.15-16】 * fix codestyle bugs * updata_ijcai_car_submodule (PaddlePaddle#882) * 【PPSCI Export&Infer No.25】bracket (PaddlePaddle#878) * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * fix api docs in the timedomain * fix api docs of timedomain * fix api docs of timedomain * ppsci api docs fixed * ppsci api docs fixed * ppsci api docs fixed * add export and infer for bracket * updata bracket doc * solve conflict according to the branch named develop * Update examples/bracket/conf/bracket.yaml * Update examples/bracket/conf/bracket.yaml * Update examples/bracket/conf/bracket.yaml * add export&inference for bracket --------- Co-authored-by: krp <2934631798@qq.com> Co-authored-by: HydrogenSulfate <490868991@qq.com> * add document of python_infer with depoly module (PaddlePaddle#885) * 【PPSCI Export&Infer No.11-12】 (PaddlePaddle#883) * fix doc bugs * fix codestyle bugs * 【PPSCI Export&Infer No.15-16】 * fix codestyle bug for PPSCI Export&Infer No.15-16】 * fix codestyle bugs for 【PPSCI Export&Infer No.15-16】 * fix codestyle bugs for 【PPSCI Export&Infer No.15-16】 * fix codestyle bugs for 【PPSCI Export&Infer No.15-16】 * fix bugs for 【PPSCI Export&Infer No.15-16】 * fix codestyle bugs * 【PPSCI Export&Infer No.11-12】 * change predictor * fix bugs in change predictor * cancel extra doc commit * fix codestyle bugs * Update examples/cylinder/2d_unsteady/cylinder2d_unsteady_Re100.py * Update examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py * Update examples/cylinder/2d_unsteady/cylinder2d_unsteady_Re100.py * Update examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py * Update examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py * cancel extra changes * cancel extra changes * update examples/cylinder/2d_unsteady/transformer_physx/conf/transformer.yaml --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * 【PPSCI Export&Infer No.21】tempoGAN (PaddlePaddle#884) * add tempoGAN export&infer * fix tempoGAN.md * fix tempoGAN.md * fix tempoGAN.py * [Doc&Refine] Add IJCAI competetion in README (PaddlePaddle#886) * add ijcai 2024 competetion information in README * refine symbolic module * Fix document(test=document_fix) * fix for pdiparams_path * [Hackathon 6th Code Camp No.15] support earthformer docs (PaddlePaddle#877) * add-earthformer * add-earthformer-doc * add-earthformer-doc * add-erthformer-doc * add-earthformer * add-earthformer-doc * add-earthformer-doc * add-earthformer-doc * add-earthformer-doc --------- Co-authored-by: zzm <95690929+zhiminzhang0830@users.noreply.github.com> * Update mkdocs.yml (PaddlePaddle#888) * 【PPSCI Export&Infer No.20】shock_wave (PaddlePaddle#890) * add shock_wave export&infer * Update examples/shock_wave/shock_wave.py * Update examples/shock_wave/shock_wave.py --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * 【PPSCI Export&Infer No.9】Bubble (PaddlePaddle#887) * 【PPSCI Export&Infer No.9】 * update examples/bubble/conf/bubble.yaml * fix codestyle bugs * Update examples/bubble/bubble.py * update examples/bubble/bubble.py --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * modify earthformer-doc (PaddlePaddle#891) * add-earthformer * add-earthformer-doc * add-earthformer-doc * add-erthformer-doc * add-earthformer * add-earthformer-doc * add-earthformer-doc * add-earthformer-doc * add-earthformer-doc * add-earthformer-doc * add-earthformer-doc --------- Co-authored-by: zzm <95690929+zhiminzhang0830@users.noreply.github.com> * 【PPSCI Export&Infer No.30】heat_exchanger (PaddlePaddle#892) * 【PPSCI Export&Infer No.30】heat_exchanger * fix codestyle bug * update examples/heat_exchanger/heat_exchanger.py * fix codestyle bugs * Update heat_exchanger.py Fix and simplify code --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * add export&infer 26-27 (PaddlePaddle#893) * [Fea] Support detach_keys argument for all PDE (PaddlePaddle#889) * fully support detach_keys argument for all PDE * add unitest for detach option * fix access for 'name' when exp do not have 'name' attribute * fix unitest * add example code for _apply_detach * fix test_pde_base * [Fix] Fix phycrnet bug (PaddlePaddle#894) * fix phycrnet bug * Update examples/phycrnet/functions.py Co-authored-by: zzm <95690929+zhiminzhang0830@users.noreply.github.com> * Update examples/phycrnet/functions.py Co-authored-by: zzm <95690929+zhiminzhang0830@users.noreply.github.com> --------- Co-authored-by: zzm <95690929+zhiminzhang0830@users.noreply.github.com> * [Doc] Update links of cooperation projects (PaddlePaddle#898) * fix(test=document_fix) * optimize importing * 添加共创计划项目链接 * [Ehn] Enhance config module (PaddlePaddle#899) * support default config content in config module and remove deprecated AttrDict series code * update corresponding unitests * update develop code * 【PPSCI Export&Infer No.13】 darcy2d (PaddlePaddle#900) * 【PPSCI Export&Infer No.13】 darcy2d * Update examples/darcy/darcy2d.py --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * Add export & inference for hPINNs (PaddlePaddle#902) * feat: add export and infer functions for hpinns * fix:register transform brfore export * 【PPSCI Export&Infer No.2】Add export & inference for DeepONet (PaddlePaddle#901) * add DeepONet export and infer * update docstring of geometry * Update deeponet.py * Update deeponet.py * [Upadte]update plotting of hpinn's inference (PaddlePaddle#903) * [Upadte]update plotting of hpinn's inference * update1 * [Fix] Fix fractional poisson 2d (PaddlePaddle#904) * Fix fractional_poisson_2d * update fractional_poisson_2d * [Doc&Fix] Update config doc and correct API (PaddlePaddle#905) * update config document and adapt viv code to new config module * update 'how to use optuna' in document * Fix * 【PPSCI Export&Infer No.35】nowcastnet (PaddlePaddle#895) * nowcastnet.py * nowcastnet.py * nowcastnet.py * Update examples/nowcastnet/nowcastnet.py Co-authored-by: HydrogenSulfate <490868991@qq.com> * Update examples/nowcastnet/nowcastnet.py Co-authored-by: HydrogenSulfate <490868991@qq.com> * nowcastnet * nowcastnet --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * Fix cylinder2d_unsteady_transformer_physx (PaddlePaddle#906) * [Fea] Add PirateNet and update allen_cahn document (PaddlePaddle#907) * Add PiraNet and update allen_cahn document * fix example code for mlp.py * rename pira to pirate * update AIStudio link for allen cahn * [Fix] Fix output_dir for visualDL and tensorboardX (PaddlePaddle#908) * fix output_dir for visualDL and tensorboardX * update scikit-learn<1.5.0 for incompatible upgrad * update docs(test=document_fix) (PaddlePaddle#909) * [Hackathon 6th Code Camp No.15] support neuraloperator (PaddlePaddle#867) * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * [Fix] Fix document requirement and update yaml (PaddlePaddle#910) * add dependencies to dynamic in pyproject.toml * Add --create-dirs in curl scripts for creating directory automatically * remove visualdl from requirements.txt * use true/false instead of True/False for compatibility * update some links * fix visualdl * [Fix] Correct desciprtion of epsilon of AllenCahn equation (PaddlePaddle#911) * Correct desciprtion of epsilon of AllenCahn equation * fix seed for test_detach * Add allen cahn sota (PaddlePaddle#879) * add allen cahn ntk * update code * add allen cahn ntk * update code * update code * update code * 修改配置 --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * 【Hackathon 6th No.39】XPINN 迁移至 PaddleScience (PaddlePaddle#849) * add XPINNs example * comment * add conf file * refine code * fix comment * fix data type * fix data type * Update examples/xpinn/plotting.py * Update examples/xpinn/conf/xpinn.yaml * refine code * refine doc * refine doc * fix doc * fix bugs --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * [Doc] Add evaluation for xpinn and add to homepage (PaddlePaddle#912) * add evaluation for xpinn and refine doc * update Examples in AllenCahn docstring * fix export and infer (PaddlePaddle#916) * update example code for PINNPredictor (PaddlePaddle#918) * auto build and upload develop pip whl to pypi when commiting (PaddlePaddle#920) * fix for 3.10 (PaddlePaddle#921) * [Refine] Refine loss and metric module (PaddlePaddle#919) * return loss dict instead of loss summation for all loss.forward * adapt all mtl module for Dict[str, Tensor] type of input losses * fix * remove 'area' in Constriant.output_keys * fix eval.py * fix code * fix examples in func.py * fix examples in func.py * Fix for MSELossWithL2Decay and train_enn.py * fix doctest in loss/mse.py * fix epnn * fix * [Doc] Refine user guide (PaddlePaddle#922) * fix code and refine userguide * print log when reach the training max_steps * update docs * [Example] Add ldc 2d Re1000 and Re3200 example (PaddlePaddle#924) * update piranet for ldc re3200 case(WIP) * add ldc_2d_Re1000 plain example * add ldc_2d_Re3200_piratenet and ldc_2d_Re3200_sota * refine code * refine docstrings * add missing blank line for docstrings * refine docstrings * replace lowercase to uppercase for first word in sentence * add ldc_2d_re3200_piratenet and ldc_2d_re3200_sota * remove redundant files * fix linenum * fix doctest * fix for allen_cahn * [Doc] Update ldc2d doc and README (PaddlePaddle#925) * update ldc2d doc * update RAEDME * 【PPSCI Export&Infer No.31】heat_pinn (PaddlePaddle#926) * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * ppsci.equation.PDE.parameters/state_dict/set_state_dict api fix * fix api docs in the timedomain * fix api docs of timedomain * fix api docs of timedomain * ppsci api docs fixed * ppsci api docs fixed * ppsci api docs fixed * add export and infer for bracket * updata bracket doc * solve conflict according to the branch named develop * Update examples/bracket/conf/bracket.yaml * Update examples/bracket/conf/bracket.yaml * Update examples/bracket/conf/bracket.yaml * add export&inference for bracket * add export and infer for heat_pinn * add export and infer for heat_pinn * Update examples/heat_pinn/heat_pinn.py * Update examples/heat_pinn/heat_pinn.py * Update examples/heat_pinn/conf/heat_pinn.yaml --------- Co-authored-by: krp <2934631798@qq.com> Co-authored-by: HydrogenSulfate <490868991@qq.com> * [Doc] Refine doc (PaddlePaddle#927) * remove vtk for slim requirements.txt * fix document of ldc2d_steady.md * refine grad_norm * remove redundant profiler.py file * refine docs * fix gradnorm * restore gradnorm to original version * remove pyvista from requirements.txt * add nightly build pip .whl url (PaddlePaddle#928) * Support gh pages (PaddlePaddle#929) * update develop mkdocs * add mike plugin to support deploying document in github pages * remove pycamotk for submodules (PaddlePaddle#930) * add workflows for github pages * update mkdocs.yml * update ai4s_universal_platform branch for gh-page * only set default to latest when current branch is develop * fix ci.yml * fix alias * use develop instead of latest * remove ci.yml in develop branch * [Fix] Fix eval (PaddlePaddle#931) * update develop mkdocs * allow alias for mike * fix and refine eval.py * fix * [Hackathon 6th Code Camp No.15] support neuraloperator docs (PaddlePaddle#917) * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator * add-neuraloperator-doc * add-neuraloperator-doc * move-paddle-harmonics * add-neuraloperator-doc * add-neuraloperator-doc * add-neuraloperator-doc * add-neuraloperator-doc * add-neuraloperator-doc * add-neuraloperator-doc * add-neuraloperator-docs * add-neuraloperator-docs * add-neuraloperator-docs * add-neuraloperator-doc * add-neuraloperator-doc * add montecarlo integrate api (PaddlePaddle#932) * add montecarlo integrate api * Update ppsci/experimental/math_module.py * Update ppsci/experimental/math_module.py * Update ppsci/experimental/math_module.py * Update ppsci/experimental/math_module.py * Update ppsci/experimental/math_module.py * fix code format error --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * 【Hackathon 6th No.37】GraphCastNet 代码迁移至 PaddleScience (PaddlePaddle#897) * add GraphGridMeshDataSet * add graphcast model and example * refine code and visualization * add license * fix bugs * fix ci errors * fix ci errors * fix ci errors * fix ci errors * fix ci errors * add docs * resolve conflicts * resolve conflicts * resolve conflicts * resolve conflicts * refine docs * fix comments * fix comments * fix comments * fix * delete atmospheric_utils.py * fix ci errors * fix * fix comments * fix * add transform in graphcast * fix model bugs * fix * fix docs * [Doc&Fix] Fix compatibility with isort and black and refine install_setup and graphcast doc (PaddlePaddle#934) * update develop mkdocs * allow alias for mike * add pymesh test notation for py310 and fix isort conflict with black * use warning instead of assert for duplicated metric name * fix amgnet for precision promotion is deprecated between int32/64 * [Doc] Refine doc and code (PaddlePaddle#935) * update develop mkdocs * allow alias for mike * fix code * use full instead of to_tensor * reset time statistics after printing * modify yinglong (PaddlePaddle#937) * modify yinglong * refine * update adr example(pre) (PaddlePaddle#938) * add Extformer-MoE example by HKUST(GZ) (PaddlePaddle#933) * add extformer-moe * fix 0624 * fix 0624 * Delete outputs_extformer_moe_pretrain directory * Delete docs/zh/examples/extformer_moe_figs directory * Update extformer_moe.md * Update extformer_moe_utils.py * Update extformer_moe.md * Update ppsci/data/dataset/__init__.py Co-authored-by: HydrogenSulfate <490868991@qq.com> * Update ppsci/data/dataset/ext_moe_enso_dataset.py Co-authored-by: HydrogenSulfate <490868991@qq.com> * fix merge review issues * fix merge 240629 * fix 240701 * fix 240701_ * fix 240701__ * fix 240701___ * Update ext_moe_enso_dataset.py * Update docs/zh/examples/extformer_moe.md Co-authored-by: HydrogenSulfate <490868991@qq.com> * Update docs/zh/examples/extformer_moe.md Co-authored-by: HydrogenSulfate <490868991@qq.com> * Update ppsci/data/dataset/ext_moe_enso_dataset.py --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> * [Fix] Fix extformer-moe (PaddlePaddle#940) * fix&refine extformer code and docs * replace paddle.nn. with nn. * update Extformer-MoE in docs * [Example] Add adv_cvit and ns_cvit (PaddlePaddle#939) * update develop mkdocs * allow alias for mike * update CViT code(WIP) * update CViT code(WIP) * update validate code * update code * update code * refine code * refine docs * update docs * update export&inference code * update ns cvit code(WIP, not aligned) * update reprod code * rename block name according to their class * add more config yamls * fix data/__init__ * fix * refine code and add more annotations * update code * update TRT steps * change pos/time embedding from buffer to trainable parameters * use interpolation for spatial_dims * remove interpolation * update config * fix std of normal initializer * update outputfile * add einops into req * restore l2_rel * refine eval checking and logging * update example code of FunctionalBatchTransform * update pretrained url and plot code * add cvit doc * update adv_cvit doc * rename title for adv_cvit.md * update docs * update docs * fix zh/examples/extformer_moe.md * update docs * refine code * [Fix] Fix batch indexing failed in phylstm2 (PaddlePaddle#941) * fix batch indexing failed in list: * fix chapter number of adv_cvit.md(test=document_fix) * [Doc] Add requirements.txt for AMGNet (PaddlePaddle#942) * remove print code * add req for amgnet * [Fix] Fix phylstm (PaddlePaddle#943) * fix phylstm running error * update version warning * fix amgnet doc(test=document_fix) * fix deepcfd * fix deepcfd * [Doc] Fix document and formulations for cvit (PaddlePaddle#944) * fix document and formulations * correct doc of Data.batch_transform * update schema for None batch size in INFER config * support batch_size to be set to None where certain data is unnecessary to be batchified * update pre-commit * refine EVAL config schema * Fix eval config (PaddlePaddle#948) * [Feature] Add loss aggregator to saving/loading process (PaddlePaddle#949) * fix document and formulations * correct doc of Data.batch_transform * update schema for None batch size in INFER config * support batch_size to be set to None where certain data is unnecessary to be batchified * update pre-commit * refine EVAL config schema * save&load state_dict for loss aggregator * [Doc] Add HKUST badge and remove some codes (PaddlePaddle#950) * add HKUST icon in cooperation institution * remove unnecessary loss_aggregator in load_pretrain function * resize image * add requests into requirements * Fix typo in mtl/base (PaddlePaddle#951) * PIDeepONet-LBM (PaddlePaddle#947) * PIDeepONet-LBM * PIDeepONet-LBM-datasetMountBOS * PIDeepONet-LBM-datasetMountBOS-CUDASet * PIDeepONet-LBM-datasetMountBOS-CUDASet * [PIR] Support pir export and infer (PaddlePaddle#952) * support export and inference in PIR mode * add validator check * fix solver.predict and device setting (PaddlePaddle#953) * fix input dtype in shock wave (PaddlePaddle#954) * [Fix] Fix dtype (PaddlePaddle#956) * align dtype of input with model parameter * wrap return training loss with dict * update aneurysm zh-cn translation: * [Fix] Adapt code to dy2st mode (PaddlePaddle#957) * shallow copy input data in expression * shallow copy in ComposedNode.forward * update tempoGAN code linenumber * update code * [Example] Add spinn on helmholtz equation (PaddlePaddle#958) * update spinn code(WIP) * update code(l2err=0.089) * update SPINN helmholtz3d * update pretraind model url(TRAIN.nc=64) * add spinn doc * refine code and docs * update * update docs * update supports information * update code and doc * refine arch code * Fix return type in examples/tempoGAN/functions.py for tempoGAN (PaddlePaddle#963) * [Add]add paddle version code of IJACA 2024 (PaddlePaddle#959) * [Add]add paddle version code of IJACA 2024 * update README.txt * update command of README.txt and some hints * [Doc] Update guidance of development (PaddlePaddle#965) * update SPINN url * update url * annotate latex which may occur error when latex font is not installed * add mtl submodule to loss.__all__ * update doc * update pre-commit guidance * update synchronization of upstream guidance * [Doc] Update url (PaddlePaddle#966) * update SPINN url * update url * annotate latex which may occur error when latex font is not installed * add mtl submodule to loss.__all__ * update doc * update pre-commit guidance * update synchronization of upstream guidance * update docs * update batch size computation * rename IJACA -> IJCAI * update description * remove redundant import * 整理代码并准备提交 * 整理代码并准备提交 * 整理代码并准备提交 * perovskite * 使用 Git LFS 跟踪大文件 * Remove large files from version control * Remove LFS tracking and update .gitattributes * 提交在 new_feature_branch 上的更改 * Your commit message * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Remove MLP folder from Git tracking * 修复 MLP_LI 文件中的尾随空格 * Save local changes to mkdocs.yml * Update MLP_LI documentation and results images * 整理代码并提交到 dev_model 分支 * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * 整理代码，提交新增和修改的文件 * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop * Resolve merge conflicts and merge upstream develop --------- Co-authored-by: HydrogenSulfate <490868991@qq.com> Co-authored-by: megemini <megemini@outlook.com> Co-authored-by: ooo oo <106524776+ooooo-create@users.noreply.github.com> Co-authored-by: Tianchi <62602289+DUCH714@users.noreply.github.com> Co-authored-by: WG <39621324+wangguan1995@users.noreply.github.com> Co-authored-by: wangguan <772359200@qq.com> Co-authored-by: xusuyong <107761330+xusuyong@users.noreply.github.com> Co-authored-by: zzm <95690929+zhiminzhang0830@users.noreply.github.com> Co-authored-by: lijialin03 <124568209+lijialin03@users.noreply.github.com> Co-authored-by: Turingg <732153168@qq.com> Co-authored-by: Turingg <117662096+Turingg@users.noreply.github.com> Co-authored-by: Tao Luo <luotao02@baidu.com> Co-authored-by: 张春乔 <83450930+Liyulingyue@users.noreply.github.com> Co-authored-by: AyaseNana <49900969+NKNaN@users.noreply.github.com> Co-authored-by: Wang Xin <xinwang614@gmail.com> Co-authored-by: GGBond8488 <33050871+GGBond8488@users.noreply.github.com> Co-authored-by: liaoxin2 <136409501+liaoxin2@users.noreply.github.com> Co-authored-by: smallpoxscattered <116335664+smallpoxscattered@users.noreply.github.com> Co-authored-by: hyDONG <116695878+1want2sleep@users.noreply.github.com> Co-authored-by: krp <2934631798@qq.com> Co-authored-by: Wu Fei <72655761+wufei2@users.noreply.github.com> Co-authored-by: MayYouBeProsperous <ljmhz@outlook.com> Co-authored-by: WoWYoYLoL <76137105+WoWYoYLoL@users.noreply.github.com> Co-authored-by: gmm <38800877+mmglove@users.noreply.github.com> Co-authored-by: Xiaoxu Chen <chenxx_id@163.com> Co-authored-by: yangchanghui <71805205+Yang-Changhui@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: KennyNH <63087259+KennyNH@users.noreply.github.com> Co-authored-by: KaiCHEN-HT <96469292+KaiCHEN-HT@users.noreply.github.com>

KennyNH and others added 9 commits June 24, 2024 13:31

add extformer-moe

cd22202

fix 0624

5efe618

fix 0624

0250709

Delete outputs_extformer_moe_pretrain directory

4ea884e

Delete docs/zh/examples/extformer_moe_figs directory

c278535

Merge branch 'PaddlePaddle:develop' into dev_model

6fcc75c

Update extformer_moe.md

6e159cc

Update extformer_moe_utils.py

8648d53

Update extformer_moe.md

a49a6fc

paddle-bot Bot added the contributor label Jun 26, 2024

HydrogenSulfate reviewed Jun 26, 2024

View reviewed changes

Comment thread ppsci/arch/extformer_moe_cuboid_utils.py Outdated

KennyNH and others added 4 commits June 26, 2024 23:36

Update ppsci/data/dataset/__init__.py

7e8a509

Co-authored-by: HydrogenSulfate <490868991@qq.com>

Update ppsci/data/dataset/ext_moe_enso_dataset.py

ef33169

Co-authored-by: HydrogenSulfate <490868991@qq.com>

Merge branch 'PaddlePaddle:develop' into dev_model

22e77db

fix merge review issues

898f949

HydrogenSulfate reviewed Jun 28, 2024

View reviewed changes

Comment thread examples/extformer_moe/extformer_moe_enso_train.py

KennyNH and others added 2 commits June 29, 2024 02:26

fix merge 240629

e7ea1c1

Merge branch 'PaddlePaddle:develop' into dev_model

687575a

KennyNH requested a review from HydrogenSulfate June 29, 2024 02:49

HydrogenSulfate requested changes Jul 1, 2024

View reviewed changes

fix 240701

dc10c5a

KennyNH requested a review from HydrogenSulfate July 1, 2024 09:28

KennyNH added 3 commits July 1, 2024 13:09

fix 240701_

625adc5

fix 240701__

b8742b2

fix 240701___

90f42ee

Merge branch 'develop' into dev_model

c67879a

HydrogenSulfate reviewed Jul 6, 2024

View reviewed changes

Comment thread ppsci/data/dataset/ext_moe_enso_dataset.py Outdated

Comment thread docs/zh/examples/extformer_moe.md Outdated

Comment thread docs/zh/examples/extformer_moe.md Outdated

KennyNH and others added 3 commits July 8, 2024 11:08

Update ext_moe_enso_dataset.py

41b6303

Update docs/zh/examples/extformer_moe.md

f795adf

Co-authored-by: HydrogenSulfate <490868991@qq.com>

Update docs/zh/examples/extformer_moe.md

71b7094

Co-authored-by: HydrogenSulfate <490868991@qq.com>

HydrogenSulfate reviewed Jul 8, 2024

View reviewed changes

Comment thread ppsci/data/dataset/ext_moe_enso_dataset.py Outdated

Update ppsci/data/dataset/ext_moe_enso_dataset.py

2bf50ac

HydrogenSulfate approved these changes Jul 8, 2024

View reviewed changes

HydrogenSulfate merged commit d39b52c into PaddlePaddle:develop Jul 8, 2024

zhiminzhang0830 mentioned this pull request Aug 13, 2024

yinglong气象大模型代码 #969

Closed

Uh oh!

Conversation

KennyNH commented Jun 26, 2024

Uh oh!

paddle-bot Bot commented Jun 26, 2024

Uh oh!

CLAassistant commented Jun 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HydrogenSulfate commented Jun 28, 2024

Uh oh!

KennyNH commented Jun 29, 2024

Uh oh!

HydrogenSulfate commented Jun 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Uh oh!

KennyNH commented Jul 1, 2024

Uh oh!

HydrogenSulfate commented Jul 1, 2024

Uh oh!

KennyNH commented Jul 1, 2024

Uh oh!

KennyNH commented Jul 1, 2024

Uh oh!

HydrogenSulfate commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KennyNH commented Jul 3, 2024 via email

Uh oh!

HydrogenSulfate commented Jul 3, 2024

Uh oh!

KennyNH commented Jul 6, 2024

Uh oh!

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KennyNH commented Jul 8, 2024

Uh oh!

HydrogenSulfate commented Jul 8, 2024

Uh oh!

Uh oh!

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Uh oh!

luotao1 commented Aug 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

CLAassistant commented Jun 26, 2024 •

edited

Loading

HydrogenSulfate commented Jun 29, 2024 •

edited

Loading

HydrogenSulfate commented Jul 3, 2024 •

edited

Loading