Dgmr #813
Merged
Dgmr #813
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
HydrogenSulfate
requested changes
Mar 25, 2024
| - mode | ||
| - output_dir | ||
| - log_freq | ||
| sweep: |
There was a problem hiding this comment.
参考其他配置文件，加上callback字段
examples/dgmr/conf/dgmr.yaml
Outdated
| split: validation # train or validation | ||
| NUM_INPUT_FRAMES: 4 | ||
| NUM_TARGET_FRAMES: 18 | ||
| dataset_path: /workspace/workspace/skillful_nowcasting/openclimatefix/nimrod-uk-1km |
examples/dgmr/conf/dgmr.yaml
Outdated
Comment on lines
38
to
42
| DATALOADER: | ||
| batch_size: 1 | ||
| shuffle: False | ||
| num_workers: 1 | ||
| drop_last: True |
There was a problem hiding this comment.
dataloader配置放py文件，不需要写在这里
Comment on lines
+46
to
+47
| input_keys: ['input_frames'] | ||
| output_keys: ['future_images'] |
examples/dgmr/dgmr.py
Outdated
|
|
||
|
|
||
| def train(cfg: DictConfig): | ||
| print("Not supported.") |
There was a problem hiding this comment.
Suggested change
| print("Not supported.") | |
| raise NotImplementedError( | |
| "Training of DGMR is not supported now." | |
| ) |
ppsci/arch/dgmr.py
Outdated
Comment on lines
948
to
955
| """ | ||
|
|
||
| Args: | ||
| x: tensor on the correct device, to move over the latent distribution | ||
|
|
||
| Returns: | ||
|
|
||
| """ |
ppsci/arch/dgmr.py
Outdated
| def __init__( | ||
| self, input_channels: int, output_channels: int, ratio_kq=8, ratio_v=8 | ||
| ): | ||
| super(AttentionLayer, self).__init__() |
There was a problem hiding this comment.
将super(XXX, self).__init__()的写法全部改为super().__init__()
ppsci/data/dataset/dgmr_dataset.py
Outdated
Comment on lines
50
to
51
| NUM_INPUT_FRAMES: int = 4, | ||
| NUM_TARGET_FRAMES: int = 18, |
ppsci/data/dataset/dgmr_dataset.py
Outdated
Comment on lines
59
to
60
| self.NUM_INPUT_FRAMES = NUM_INPUT_FRAMES | ||
| self.NUM_TARGET_FRAMES = NUM_TARGET_FRAMES |
ppsci/data/dataset/dgmr_dataset.py
Outdated
| def __len__(self): | ||
| return self.number | ||
|
|
||
| def __getitem__(self, item): |
HydrogenSulfate
requested changes
Mar 26, 2024
ppsci/arch/dgmr.py
Outdated
Comment on lines
407
to
415
| """ | ||
| Spatial discriminator from Skillful Nowcasting, see https://arxiv.org/pdf/2104.00954.pdf | ||
|
|
||
| Args: | ||
| input_channels: Number of input channels per timestep | ||
| num_timesteps: Number of timesteps to use, in the paper 8/18 timesteps were chosen | ||
| num_layers: Number of intermediate DBlock layers to use | ||
| conv_type: Type of 2d convolutions to use, see satflow/models/utils.py for options | ||
| """ |
There was a problem hiding this comment.
__init__方法的 docstring 移动到SpatialDiscriminator下方
There was a problem hiding this comment.
class的__init__方法的 docstring 移动到class的下方
HydrogenSulfate
requested changes
Mar 26, 2024
There was a problem hiding this comment.
docstring格式不规范，重新用vscode插件生成下
ppsci/arch/dgmr.py
Outdated
Comment on lines
12
to
15
| raise ModuleNotFoundError( | ||
| "Please install einops with 'pip install einops'" | ||
| " before exporting DGMR model." | ||
| ) |
ppsci/arch/dgmr.py
Outdated
Comment on lines
321
to
329
| """ | ||
| Temporal Discriminator from the Skillful Nowcasting, see https://arxiv.org/pdf/2104.00954.pdf | ||
|
|
||
| Args: | ||
| input_channels: Number of channels per timestep | ||
| crop_size: Size of the crop, in the paper half the width of the input images | ||
| num_layers: Number of intermediate DBlock layers to use | ||
| conv_type: Type of 2d convolutions to use, see satflow/models/utils.py for options | ||
| """ |
There was a problem hiding this comment.
python类的__init__的docstring，统一放到类下方，而不是__init__下方
ppsci/arch/dgmr.py
Outdated
|
|
||
| def forward(self, x: paddle.Tensor) -> paddle.Tensor: | ||
| """ | ||
|
|
ppsci/data/dataset/dgmr_dataset.py
Outdated
|
|
||
| import numpy as np | ||
| import paddle | ||
| from datasets import load_dataset |
There was a problem hiding this comment.
不要从模块中直接导入方法，而是通过导入模块再调用，改为像einops一样用try导入 import datasets
ppsci/data/dataset/dgmr_dataset.py
Outdated
| self.num_input_frames = num_input_frames | ||
| self.num_target_frames = num_target_frames | ||
| self.number = number | ||
| self.reader = load_dataset( |
There was a problem hiding this comment.
datasets.load_dataset
HydrogenSulfate
requested changes
Mar 26, 2024
docs/zh/examples/dgmr.md
Outdated
Comment on lines
3
to
13
| === "模型训练命令" | ||
|
|
||
| ``` sh | ||
| python dgmr.py | ||
| ``` | ||
|
|
||
| === "模型评估命令" | ||
|
|
||
| ``` sh | ||
| python dgmr.py mode=eval EVAL.pretrained_model_path=https://paddle-org.bj.bcebos.com/paddlescience/models/dgmr/dgmr_pretrained.pdparams | ||
| ``` |
There was a problem hiding this comment.
训练和验证命令补充数据集下载的脚本，否则用户还需要手动去找数据集怎么下载
HydrogenSulfate
requested changes
Mar 27, 2024
| ) | ||
| plt.subplots_adjust(hspace=0.1, wspace=0.1) | ||
| plt.savefig(osp.join(output_dir, f"Generated_Image_Frame_{batch_idx}.png")) | ||
|
|
HydrogenSulfate
requested changes
Mar 27, 2024
examples/dgmr/dgmr.py
Outdated
| cfg: DictConfig, | ||
| solver: ppsci.solver.Solver, | ||
| batch: tuple, | ||
| batch_idx: int, |
examples/dgmr/dgmr.py
Outdated
| def validation( | ||
| cfg: DictConfig, | ||
| solver: ppsci.solver.Solver, | ||
| batch: tuple, |
There was a problem hiding this comment.
- 使用Tuple而不是tuple
- 写清楚Tuple里的内容
examples/dgmr/dgmr.py
Outdated
| Args: | ||
| cfg (DictConfig): Configuration object. | ||
| solver (ppsci.solver.Solver): Solver object containing the model and related components. | ||
| batch (tuple): Input batch consisting of images and corresponding future images. |
ppsci/data/dataset/dgmr_dataset.py
Outdated
| try: | ||
| row = next(self.iter_reader) | ||
| except Exception: | ||
| rng = default_rng() |
There was a problem hiding this comment.
这里得固定下seed，不然会有随机
HydrogenSulfate
requested changes
Mar 28, 2024
ppsci/data/dataset/dgmr_dataset.py
Outdated
|
|
||
| Examples: | ||
| >>> import ppsci | ||
| >>> dataset = ppsci.data.dataset.DGMRDataset(("input", ), ("output", )) |
There was a problem hiding this comment.
45行末尾加上： # doctest: +SKIP
HydrogenSulfate
requested changes
Mar 29, 2024
ppsci/arch/dgmr.py
Outdated
Comment on lines
44
to
45
| >>> output_dict = model(input_dict) | ||
| >>> print(output_dict["output"].shape) |
There was a problem hiding this comment.
末尾加上： # doctest: +SKIP
huohuohuohuohuo123
pushed a commit
to huohuohuohuohuo123/PaddleScience
that referenced
this pull request
Aug 12, 2024
* add bubble datafile test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code test=develop * add bubble code * add bubble data * add bubble code * add some code for chip * add some code for DGMR * add some code for DGMR * add some code for DGMR * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code * add some dgmr code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
One of dgmr eval code
PR changes
One of APIs
Describe
dgmr eval code