diff --git a/README.md b/README.md index 6b7dd28c1..2fea7e524 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -
+
# MindCV @@ -26,77 +26,43 @@ English | [中文](README_CN.md)
## Introduction + MindCV is an open-source toolbox for computer vision research and development based on [MindSpore](https://www.mindspore.cn/en). It collects a series of classic and SoTA vision models, such as ResNet and SwinTransformer, along with their pre-trained weights and training strategies. SoTA methods such as auto augmentation are also provided for performance improvement. With the decoupled module design, it is easy to apply or adapt MindCV to your own CV tasks. -
- Major Features +### Major Features - **Easy-to-Use.** MindCV decomposes the vision framework into various configurable components. It is easy to customize your data pipeline, models, and learning pipeline with MindCV: -```python ->>> import mindcv -# create a dataset ->>> dataset = mindcv.create_dataset('cifar10', download=True) -# create a model ->>> network = mindcv.create_model('resnet50', pretrained=True) -``` + ```pycon + >>> import mindcv + # create a dataset + >>> dataset = mindcv.create_dataset('cifar10', download=True) + # create a model + >>> network = mindcv.create_model('resnet50', pretrained=True) + ``` -Users can customize and launch their transfer learning or training task in one command line. + Users can customize and launch their transfer learning or training task in one command line. -``` python -# transfer learning in one command line ->>> !python train.py --model=swin_tiny --pretrained --opt=adamw --lr=0.001 --data_dir={data_dir} -``` + ```shell + # transfer learning in one command line + python train.py --model=swin_tiny --pretrained --opt=adamw --lr=0.001 --data_dir=/path/to/data + ``` - **State-of-The-Art.** MindCV provides various CNN-based and Transformer-based vision models including SwinTransformer. Their pretrained weights and performance reports are provided to help users select and reuse the right model: -- **Flexibility and efficiency.** MindCV is built on MindSpore which is an efficent DL framework that can be run on different hardware platforms (GPU/CPU/Ascend). It supports both graph mode for high efficiency and pynative mode for flexibility. - - -
+- **Flexibility and efficiency.** MindCV is built on MindSpore which is an efficient DL framework that can be run on different hardware platforms (GPU/CPU/Ascend). It supports both graph mode for high efficiency and pynative mode for flexibility. -### Benchmark Results +## Model Zoo -The performance of the models trained with MindCV is summarized in [benchmark_results.md](./benchmark_results.md), where the training recipes and weights are both available. +The performance of the models trained with MindCV is summarized in [here](https://mindspore-lab.github.io/mindcv/modelzoo/), where the training recipes and weights are both available. -Model introduction and training details can be viewed in each subfolder under [configs](configs). +Model introduction and training details can be viewed in each sub-folder under [configs](configs). ## Installation -### Dependency - -- mindspore >= 1.8.1 -- numpy >= 1.17.0 -- pyyaml >= 5.3 -- tqdm -- openmpi 4.0.3 (for distributed mode) - -To install the dependency, please run -```shell -pip install -r requirements.txt -``` - -MindSpore can be easily installed by following the official [instructions](https://www.mindspore.cn/install) where you can select your hardware platform for the best fit. To run in distributed mode, [openmpi](https://www.open-mpi.org/software/ompi/v4.0/) is required to install. - -The following instructions assume the desired dependency is fulfilled. +See [Installation](https://mindspore-lab.github.io/mindcv/installation/) for details. -### Install with PyPI - -The released version of MindCV can be installed via `PyPI` as follows: -```shell -pip install mindcv -``` - -### Install from Source - -The latest version of MindCV can be installed as follows: -```shell -pip install git+https://github.com/mindspore-lab/mindcv.git -``` - -> Notes: MindCV can be installed on Linux and Mac but not on Windows currently. - -## Get Started +## Getting Started ### Hands-on Tutorial @@ -104,7 +70,7 @@ To get started with MindCV, please see the [Quick Start](docs/en/tutorials/quick Below are a few code snippets for your taste. -```python +```pycon >>> import mindcv # List and find a pretrained vision model >>> mindcv.list_models("swin*", pretrained=True) @@ -118,17 +84,19 @@ Below are a few code snippets for your taste. **Image classification demo** +Right click on the image below and save as `dog.jpg`. +

-Infer the input image with a pretrained SoTA model, +Classify the downloaded image with a pretrained SoTA model: -```python ->>> !python infer.py --model=swin_tiny --image_path='./tutorials/data/test/dog/dog.jpg' +```pycon +>>> !python infer.py --model=swin_tiny --image_path='./dog.jpg' {'Labrador retriever': 0.5700152, 'golden retriever': 0.034551315, 'kelpie': 0.010108651, 'Chesapeake Bay retriever': 0.008229004, 'Walker hound, Walker foxhound': 0.007791956} ``` -The top-1 prediction result is labrador retriever (拉布拉多犬), which is the breed of this cut dog. +The top-1 prediction result is labrador retriever, which is the breed of this cut dog. ### Training @@ -136,85 +104,92 @@ It is easy to train your model on a standard or customized dataset using `train. - Standalone Training -``` shell -# standalone training -python train.py --model=resnet50 --dataset=cifar10 --dataset_download -``` + ```shell + # standalone training + python train.py --model=resnet50 --dataset=cifar10 --dataset_download + ``` -Above is an example for training ResNet50 on CIFAR10 dataset on a CPU/GPU/Ascend device + Above is an example for training ResNet50 on CIFAR10 dataset on a CPU/GPU/Ascend device - Distributed Training -For large datasets like ImageNet, it is necessary to do training in distributed mode on multiple devices. This can be achieved with `mpirun` and parallel features supported by MindSpore. + For large datasets like ImageNet, it is necessary to do training in distributed mode on multiple devices. This can be achieved with `mpirun` and parallel features supported by MindSpore. -```shell -# distributed training -# assume you have 4 GPUs/NPUs -mpirun -n 4 python train.py --distribute \ - --model=densenet121 --dataset=imagenet --data_dir=/path/to/imagenet -``` -> Notes: If the script is executed by the root user, the `--allow-run-as-root` parameter must be added to `mpirun`. + ```shell + # distributed training + # assume you have 4 GPUs/NPUs + mpirun -n 4 python train.py --distribute \ + --model=densenet121 --dataset=imagenet --data_dir=/path/to/imagenet + ``` + > Notes: If the script is executed by the root user, the `--allow-run-as-root` parameter must be added to `mpirun`. -Detailed parameter definitions can be seen in `config.py` and checked by running `python train.py --help'. + Detailed parameter definitions can be seen in `config.py` and checked by running `python train.py --help'. -To resume training, please set the `--ckpt_path` and `--ckpt_save_dir` arguments. The optimizer state including the learning rate of the last stopped epoch will also be recovered. + To resume training, please set the `--ckpt_path` and `--ckpt_save_dir` arguments. The optimizer state including the learning rate of the last stopped epoch will also be recovered. - Config and Training Strategy -You can configure your model and other components either by specifying external parameters or by writing a yaml config file. Here is an example of training using a preset yaml file. + You can configure your model and other components either by specifying external parameters or by writing a yaml config file. Here is an example of training using a preset yaml file. -```shell -mpirun --allow-run-as-root -n 4 python train.py -c configs/squeezenet/squeezenet_1.0_gpu.yaml -``` + ```shell + mpirun --allow-run-as-root -n 4 python train.py -c configs/squeezenet/squeezenet_1.0_gpu.yaml + ``` -**Pre-defined Training Strategies:** We provide more than 20 training recipes that achieve SoTA results on ImageNet currently. Please look into the [`configs`](configs) folder for details. Please feel free to adapt these training strategies to your own model for performance improvement， which can be easily done by modifying the yaml file. + **Pre-defined Training Strategies:** + We provide more than 20 training recipes that achieve SoTA results on ImageNet currently. + Please look into the [`configs`](configs) folder for details. + Please feel free to adapt these training strategies to your own model for performance improvement, which can be easily done by modifying the yaml file. - Train on ModelArts/OpenI Platform -To run training on the [ModelArts](https://www.huaweicloud.com/intl/en-us/product/modelarts.html) or [OpenI](https://openi.pcl.ac.cn/) cloud platform: + To run training on the [ModelArts](https://www.huaweicloud.com/intl/en-us/product/modelarts.html) or [OpenI](https://openi.pcl.ac.cn/) cloud platform: + + ```text + 1. Create a new training task on the cloud platform. + 2. Add run parameter `config` and specify the path to the yaml config file on the website UI interface. + 3. Add run parameter `enable_modelarts` and set True on the website UI interface. + 4. Fill in other blanks on the website and launch the training task. + ``` + +**Graph Mode and PyNative Mode**: + +By default, the training pipeline `train.py` is run in [graph mode](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html#%E9%9D%99%E6%80%81%E5%9B%BE) on MindSpore, which is optimized for efficiency and parallel computing with a compiled static graph. +In contrast, [pynative mode](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html#%E5%8A%A8%E6%80%81%E5%9B%BE) is optimized for flexibility and easy debugging. You may alter the parameter `--mode` to switch to pure pynative mode for debugging purpose. + +**Mixed Mode**: + +[Pynative mode with ms_function ](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/combine.html) is a mixed mode for comprising flexibility and efficiency in MindSpore. To apply pynative mode with ms_function for training, please run `train_with_func.py`, e.g., -```text -1. Create a new training task on the cloud platform. -2. Add run parameter `config` and specify the path to the yaml config file on the website UI interface. -3. Add run parameter `enable_modelarts` and set True on the website UI interface. -4. Fill in other blanks on the website and launch the training task. +```shell +python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download --epoch_size=10 ``` +> Note: this is an **experimental** function under improvement. It is not stable on MindSpore 1.8.1 or earlier versions. + ### Validation -To evalute the model performance, please run `validate.py` +To evaluate the model performance, please run `validate.py` ```shell # validate a trained checkpoint python validate.py --model=resnet50 --dataset=imagenet --data_dir=/path/to/data --ckpt_path=/path/to/model.ckpt ``` -- Validation while Training +***Validation while Training*** You can also track the validation accuracy during training by enabling the `--val_while_train` option. ```shell python train.py --model=resnet50 --dataset=cifar10 \ - --val_while_train --val_split=test --val_interval=1 + --val_while_train --val_split=test --val_interval=1 ``` The training loss and validation accuracy for each epoch will be saved in `{ckpt_save_dir}/results.log`. -More examples about training and validation can be seen in [examples/scripts](examples/scripts). - -- Graph Mode and Pynative Mode - -By default, the training pipeline `train.py` is run in [graph mode](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html#%E9%9D%99%E6%80%81%E5%9B%BE) on MindSpore, which is optimized for efficiency and parallel computing with a compiled static graph. In contrast, [pynative mode](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html#%E5%8A%A8%E6%80%81%E5%9B%BE) is optimized for flexibility and easy debugging. You may alter the parameter `--mode` to switch to pure pynative mode for debugging purpose. - -[Pynative mode with ms_function ](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/combine.html) is a mixed mode for comprising flexibility and efficiency in MindSpore. To apply pynative mode with ms_function for training, please run `train_with_func.py`, e.g., - -``` shell -python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download --epoch_size=10 -``` ->Note: this is an **experimental** function under improvement. It is not stable on MindSpore 1.8.1 or earlier versions. - +More examples about training and validation can be seen in [examples](examples/scripts). ## Tutorials + We provide the following jupyter notebook tutorials to help users learn to use MindCV. - [Learn about configs](docs/en/tutorials/configuration.md) @@ -228,7 +203,7 @@ We provide the following jupyter notebook tutorials to help users learn to use M Currently, MindCV supports the model families listed below. More models with pre-trained weights are under development and will be released soon. -
+
Supported models * Big Transfer ResNetV2 (BiT) - https://arxiv.org/abs/1912.11370 @@ -272,7 +247,8 @@ Please see [configs](./configs) for the details about model performance and pret
## Supported Algorithms -
+ +
Supported algorithms * Augmentation @@ -281,12 +257,12 @@ Please see [configs](./configs) for the details about model performance and pret * [Repeated Augmentation](https://openaccess.thecvf.com/content_CVPR_2020/papers/Hoffer_Augment_Your_Batch_Improving_Generalization_Through_Instance_Repetition_CVPR_2020_paper.pdf) * RandErasing (Cutout) * CutMix - * Mixup + * MixUp * RandomResizeCrop * Color Jitter, Flip, etc * Optimizer * Adam - * Adamw + * AdamW * [Lion](https://arxiv.org/abs/2302.06675) * Adan (experimental) * AdaGrad @@ -312,94 +288,45 @@ Please see [configs](./configs) for the details about model performance and pret * Soft Binary Cross Entropy Loss (automatically enabled if mixup or label smoothing is used) * Ensemble * Warmup EMA (Exponential Moving Average) +
-## Notes -### What is New -- 2023/04/28 -1. Add some new models, listed as following - - [VGG](configs/vgg) - - [DPN](configs/dpn) - - [ResNet v2](configs/resnetv2) - - [MnasNet](configs/mnasnet) - - [MixNet](configs/mixnet) - - [RepVGG](configs/repvgg) - - [ConvNeXt](configs/convnext) - - [Swin Transformer](configs/swintransformer) - - [EdgeNeXt](configs/edgenext) - - [CrossViT](configs/crossvit) - - [XCiT](configs/xcit) - - [CoAT](configs/coat) - - [PiT](configs/pit) - - [PVT v2](configs/pvt_v2) - - [MobileViT](configs/mobilevit) -2. Bug fix: - - Setting the same random seed for each rank - - Checking if options from yaml config exist in argument parser - - Initializing flag variable as `Tensor` in Optimizer `Adan` - -- 2023/03/25 -1. Update checkpoints for pretrained ResNet for better accuracy - - ResNet18 (from 70.09 to 70.31 @Top1 accuracy) - - ResNet34 (from 73.69 to 74.15 @Top1 accuracy) - - ResNet50 (from 76.64 to 76.69 @Top1 accuracy) - - ResNet101 (from 77.63 to 78.24 @Top1 accuracy) - - ResNet152 (from 78.63 to 78.72 @Top1 accuracy) -2. Rename checkpoint file name to follow naming rule ({model_scale-sha256sum.ckpt}) and update download URLs. - -- 2023/03/05 -1. Add Lion (EvoLved Sign Momentum) optimizer from paper https://arxiv.org/abs/2302.06675 - - To replace adamw with lion, LR is usually 3-10x smaller, and weight decay is usually 3-10x larger than adamw. -2. Add 6 new models with training recipes and pretrained weights for - - [HRNet](configs/hrnet) - - [SENet](configs/senet) - - [GoogLeNet](configs/googlenet) - - [Inception V3](configs/inception_v3) - - [Inception V4](configs/inception_v4) - - [Xception](configs/xception) -3. Support gradient clip -4. Arg name `use_ema` changed to **`ema`**, add `ema: True` in yaml to enable EMA. - -- 2023/01/10 -1. MindCV v0.1 released! It can be installed via PyPI `pip install mindcv` now. -2. Add training recipe and trained weights of googlenet, inception_v3, inception_v4, xception - -- 2022/12/09 -1. Support lr warmup for all lr scheduling algorithms besides cosine decay. -2. Add repeated augmentation, which can be enabled by setting `--aug_repeats` to be a value larger than 1 (typically, 3 or 4 is a common choice). -3. Add EMA. -4. Improve BCE loss to support mixup/cutmix. - -- 2022/11/21 -1. Add visualization for loss and acc curves -2. Support epochwise lr warmup cosine decay (previous is stepwise) -- 2022/11/09 -1. Add 7 pretrained ViT models. -2. Add RandAugment augmentation. -3. Fix CutMix efficiency issue and CutMix and Mixup can be used together. -4. Fix lr plot and scheduling bug. -- 2022/10/12 -1. Both BCE and CE loss now support class-weight config, label smoothing, and auxiliary logit input (for networks like inception). -- 2022/09/13 -1. Add Adan optimizer (experimental) - -### How to Contribute +## What is New + +- 2023/5/30 +1. New Models: + - AMP(O2) version of [VGG](configs/vgg) + - [GhostNet](configs/ghostnet) + - AMP(O3) version of [MobileNetV2](configs/mobilenetv2) and [MobileNetV3](configs/mobilenetv3) + - (x,y)_(200,400,600,800)mf of [RegNet](configs/regnet) + - b1g2, b1g4 & b2g4 of [RepVGG](configs/repvgg) + - 0.5 of [MnasNet](configs/mnasnet) + - b3 & b4 of [PVTv2](configs/pvt_v2) +2. New Features: + - 3-Augment, Augmix, TrivialAugmentWide +3. Bug Fixes: + - ViT pooling mode + +See [RELEASE](RELEASE.md) for detailed history. + +## How to Contribute We appreciate all kind of contributions including issues and PRs to make MindCV better. -Please refer to [CONTRIBUTING.md](CONTRIBUTING.md) for the contributing guideline. Please follow the [Model Template and Guideline](docs/en/how_to_guides/write_a_new_model.md) for contributing a model that fits the overall interface :) +Please refer to [CONTRIBUTING.md](CONTRIBUTING.md) for the contributing guideline. +Please follow the [Model Template and Guideline](docs/en/how_to_guides/write_a_new_model.md) for contributing a model that fits the overall interface :) -### License +## License This project follows the [Apache License 2.0](LICENSE.md) open-source license. -### Acknowledgement +## Acknowledgement MindCV is an open-source project jointly developed by the MindSpore team, Xidian University, and Xi'an Jiaotong University. Sincere thanks to all participating researchers and developers for their hard work on this project. We also acknowledge the computing resources provided by [OpenI](https://openi.pcl.ac.cn/). -### Citation +## Citation If you find this project useful in your research, please consider citing: diff --git a/README_CN.md b/README_CN.md index 7405d2c68..88b3a60cb 100644 --- a/README_CN.md +++ b/README_CN.md @@ -1,4 +1,4 @@ -
+
# MindCV @@ -27,80 +27,40 @@ ## 简介 -MindCV是一个基于 [MindSpore](https://www.mindspore.cn/) -开发的，致力于计算机视觉相关技术研发的开源工具箱。它提供大量的计算机视觉领域的经典模型和SoTA模型以及它们的预训练权重和训练策略。同时，还提供了自动增强等SoTA算法来提高模型性能。通过解耦的模块设计，您可以轻松地将MindCV应用到您自己的CV任务中。 +MindCV是一个基于 [MindSpore](https://www.mindspore.cn/) 开发的，致力于计算机视觉相关技术研发的开源工具箱。它提供大量的计算机视觉领域的经典模型和SoTA模型以及它们的预训练权重和训练策略。同时，还提供了自动增强等SoTA算法来提高模型性能。通过解耦的模块设计，您可以轻松地将MindCV应用到您自己的CV任务中。 -
- 主要特性 +### 主要特性 - **高易用性** MindCV将视觉任务分解为各种可配置的组件，用户可以轻松地构建自己的数据处理和模型训练流程。 -```python ->>> import mindcv -# 创建数据集 ->>> dataset = mindcv.create_dataset('cifar10', download=True) -# 创建模型 ->>> network = mindcv.create_model('resnet50', pretrained=True) -``` + ```pycon + >>> import mindcv + # 创建数据集 + >>> dataset = mindcv.create_dataset('cifar10', download=True) + # 创建模型 + >>> network = mindcv.create_model('resnet50', pretrained=True) + ``` -用户可通过预定义的训练和微调脚本，快速配置并完成训练或迁移学习任务。 + 用户可通过预定义的训练和微调脚本，快速配置并完成训练或迁移学习任务。 -```shell -# 配置和启动迁移学习任务 -python train.py --model swin_tiny --pretrained --opt=adamw --lr=0.001 --data_dir=/path/to/dataset -``` + ```shell + # 配置和启动迁移学习任务 + python train.py --model swin_tiny --pretrained --opt=adamw --lr=0.001 --data_dir=/path/to/dataset + ``` - **高性能** MindCV集成了大量基于CNN和和Transformer的高性能模型, 如SwinTransformer，并提供预训练权重、训练策略和性能报告，帮助用户快速选型并将其应用于视觉模型。 - **灵活高效** MindCV基于高效的深度学习框架MindSpore开发，具有自动并行和自动微分等特性，支持不同硬件平台上（CPU/GPU/Ascend），同时支持效率优化的静态图模式和调试灵活的动态图模式。 -
- -### 性能结果 +## 模型支持 - -基于MindCV进行模型实现和重训练的汇总结果详见[benchmark_results.md](./benchmark_results.md), 所用到的训练策略和训练后的模型权重均可通过表中链接获取。 +基于MindCV进行模型实现和重训练的汇总结果详见[模型仓库](https://mindspore-lab.github.io/mindcv/zh/modelzoo/), 所用到的训练策略和训练后的模型权重均可通过表中链接获取。 各模型讲解和训练说明详见[configs](configs)目录。 - ## 安装 -### 依赖 - -- mindspore >= 1.8.1 -- numpy >= 1.17.0 -- pyyaml >= 5.3 -- tqdm -- openmpi 4.0.3 (分布式模式需要使用) - -运行以下脚本，安装相关依赖。 - -```shell -pip install -r requirements.txt -``` - -用户可遵从[官方指导](https://www.mindspore.cn/install) 并根据自身使用的硬件平台选择最适合您的MindSpore版本来进行安装。如果需要在分布式条件下使用，还需安装[openmpi](https://www.open-mpi.org/software/ompi/v4.0/) 。 - -之后的说明将默认用户已正确安装好相关依赖。 - -### PyPI安装 - -MindCV的已发布版本可以通过PyPI安装。 - -```shell -pip install mindcv -``` - -### 源码安装 - -Git上最新的MindCV可以通过以下指令安装。 - -```shell -pip install git+https://github.com/mindspore-lab/mindcv.git -``` - -> 注：MindCV可以在Linux和Mac系统安装，但是目前还不能在Windows系统上安装。 +详情请见[安装](https://mindspore-lab.github.io/mindcv/zh/installation/)页面 ## 快速入门 @@ -110,7 +70,7 @@ pip install git+https://github.com/mindspore-lab/mindcv.git 以下是一些供您快速体验的代码样例。 -```python +```pycon >>> import mindcv # 列出满足条件的预训练模型名称 >>> mindcv.list_models("swin*", pretrained=True) @@ -124,16 +84,17 @@ pip install git+https://github.com/mindspore-lab/mindcv.git **图片分类示例** +右键点击如下图片，另存为`dog.jpg`。 +

-使用加载了预训练参数的SoTA模型对一张图片进行推理。 +使用加载了预训练参数的SoTA模型对图片进行推理。 -```python ->>> !python infer.py - -model = swin_tiny - -image_path = './tutorials/data/test/dog/dog.jpg' -{'Labrador retriever': 0.5700152, 'golden retriever': 0.034551315, 'kelpie': 0.010108651, - 'Chesapeake Bay retriever': 0.008229004, 'Walker hound, Walker foxhound': 0.007791956} +```pycon +>>> !python infer.py --model=swin_tiny --image_path='./dog.jpg' +{'Labrador retriever': 0.5700152, 'golden retriever': 0.034551315, 'kelpie': 0.010108651, 'Chesapeake Bay retriever': 0.008229004, 'Walker hound, Walker foxhound': 0.007791956} ``` 预测结果排名前1的是拉布拉多犬，正是这张图片里的狗狗的品种。 @@ -144,51 +105,66 @@ pip install git+https://github.com/mindspore-lab/mindcv.git - 单卡训练 -```shell -# 单卡训练 -python train.py --model resnet50 --dataset cifar10 --dataset_download -``` + ```shell + # 单卡训练 + python train.py --model resnet50 --dataset cifar10 --dataset_download + ``` -以上代码是在CIFAR10数据集上单卡（CPU/GPU/Ascend）训练ResNet的示例，通过`model`和`dataset`参数分别指定需要训练的模型和数据集。 + 以上代码是在CIFAR10数据集上单卡（CPU/GPU/Ascend）训练ResNet的示例，通过`model`和`dataset`参数分别指定需要训练的模型和数据集。 - 分布式训练 -对于像ImageNet这样的大型数据集，有必要在多个设备上以分布式模式进行训练。基于MindSpore对分布式相关功能的良好支持，用户可以使用`mpirun`来进行模型的分布式训练。 - -```shell -# 分布式训练 -# 假设你有4张GPU或者NPU卡 -mpirun --allow-run-as-root -n 4 python train.py --distribute \ - --model densenet121 --dataset imagenet --data_dir ./datasets/imagenet -``` + 对于像ImageNet这样的大型数据集，有必要在多个设备上以分布式模式进行训练。基于MindSpore对分布式相关功能的良好支持，用户可以使用`mpirun`来进行模型的分布式训练。 -完整的参数列表及说明在`config.py`中定义，可运行`python train.py --help`快速查看。 + ```shell + # 分布式训练 + # 假设你有4张GPU或者NPU卡 + mpirun --allow-run-as-root -n 4 python train.py --distribute \ + --model densenet121 --dataset imagenet --data_dir ./datasets/imagenet + ``` -如需恢复训练，请指定`--ckpt_path`和`--ckpt_save_dir`参数，脚本将加载路径中的模型权重和优化器状态，并恢复中断的训练进程。 + 完整的参数列表及说明在`config.py`中定义，可运行`python train.py --help`快速查看。 + 如需恢复训练，请指定`--ckpt_path`和`--ckpt_save_dir`参数，脚本将加载路径中的模型权重和优化器状态，并恢复中断的训练进程。 - 超参配置和预训练策略 + 您可以编写yaml文件或设置外部参数来指定配置数据、模型、优化器等组件及其超参。以下是使用预设的训练策略（yaml文件）进行模型训练的示例。 -您可以编写yaml文件或设置外部参数来指定配置数据、模型、优化器等组件及其超参。以下是使用预设的训练策略（yaml文件）进行模型训练的示例。 + ```shell + mpirun --allow-run-as-root -n 4 python train.py -c configs/squeezenet/squeezenet_1.0_gpu.yaml + ``` -```shell -mpirun --allow-run-as-root -n 4 python train.py -c configs/squeezenet/squeezenet_1.0_gpu.yaml -``` +**预定义的训练策略** +MindCV目前提前了超过20种模型训练策略，在ImageNet取得SoTA性能。 +具体的参数配置和详细精度性能汇总请见[`configs`](configs)文件夹。 +您可以便捷将这些训练策略用于您的模型训练中以提高性能（复用或修改相应的yaml文件即可） -**预定义的训练策略** MindCV目前提前了超过20种模型训练策略，在ImageNet取得SoTA性能。具体的参数配置和详细精度性能汇总请见[`configs`](configs)文件夹。您可以便捷将这些训练策略用于您的模型训练中以提高性能（复用或修改相应的yaml文件即可） +- 在ModelArts/OpenI平台上训练 + 在[ModelArts](https://www.huaweicloud.com/intl/en-us/product/modelarts.html)或[OpenI](https://openi.pcl.ac.cn/)云平台上进行训练，需要执行以下操作，： -- 在ModelArts/OpenI平台上训练 + ```text + 1、在云平台上创建新的训练任务。 + 2、在网站UI界面添加运行参数`config`，并指定yaml配置文件的路径。 + 3、在网站UI界面添加运行参数`enable_modelarts`并设置为True。 + 4、在网站上填写其他训练信息并启动培训任务。 + ``` -在[ModelArts](https://www.huaweicloud.com/intl/en-us/product/modelarts.html)或[OpenI](https://openi.pcl.ac.cn/)云平台上进行训练，需要执行以下操作，： +**静态图和动态图模式** +在默认情况下，模型训练（`train.py`）在MindSpore上以[图模式](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html) 运行，该模式对使用静态图编译对性能和并行计算进行了优化。 +相比之下，[pynative模式](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html#%E5%8A%A8%E6%80%81%E5%9B%BE)的优势在于灵活性和易于调试。为了方便调试，您可以将参数`--mode`设为1以将运行模式设置为调试模式。 + +**混合模式** + +[基于ms_function的混合模式](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/combine.html) 是兼顾了MindSpore的效率和灵活的混合模式。用户可通过使用`train_with_func.py`文件来使用该混合模式进行训练。 + +```shell +python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download --epoch_size=10 ``` -1、在云平台上创建新的训练任务。 -2、在网站UI界面添加运行参数`config`，并指定yaml配置文件的路径。 -3、在网站UI界面添加运行参数`enable_modelarts`并设置为True。 -4、在网站上填写其他训练信息并启动培训任务。 -``` + +> 注：此为试验性质的训练脚本，仍在改进，在1.8.1或更早版本的MindSpore上使用此模式目前并不稳定。 ### 模型验证 @@ -199,30 +175,18 @@ mpirun --allow-run-as-root -n 4 python train.py -c configs/squeezenet/squeezenet python validate.py --model=resnet50 --dataset=imagenet --data_dir=/path/to/data --ckpt_path=/path/to/model.ckpt ``` -- 训练过程中进行验证 +**训练过程中进行验证** 当需要在训练过程中，跟踪模型在测试集上精度的变化时，请启用参数`--val_while_train`，如下 ```shell python train.py --model=resnet50 --dataset=cifar10 \ - --val_while_train --val_split=test --val_interval=1 + --val_while_train --val_split=test --val_interval=1 ``` 各轮次的训练损失和测试精度将保存在`{ckpt_save_dir}/results.log`中。 - -- 静态图和动态图模式 - -在默认情况下，模型训练（`train.py`）在MindSpore上以[图模式](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html) 运行，该模式对使用静态图编译对性能和并行计算进行了优化。相比之下，[pynative模式](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/mode.html#%E5%8A%A8%E6%80%81%E5%9B%BE)的优势在于灵活性和易于调试。 -为了方便调试，您可以将参数`--mode`设为1以将运行模式设置为调试模式。 - -[基于ms_function的混合模式](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/combine.html) 是兼顾了MindSpore的效率和灵活的混合模式。用户可通过使用`train_with_func.py`文件来使用该混合模式进行训练。 - -```shell -python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download --epoch_size=10 -``` - -> 注：此为试验性质的训练脚本，仍在改进，在1.8.1或更早版本的MindSpore上使用此模式目前并不稳定。 +更多训练和验证的示例请见[示例](examples/scripts)。 ## 教程 @@ -232,14 +196,14 @@ python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download - [模型推理](docs/zh/tutorials/inference.md) - [自定义数据集上的模型微调训练](docs/zh/tutorials/finetune.md) - [如何自定义模型]() //coming soon -- [视觉ransformer性能优化]() //coming soon +- [视觉transformer性能优化]() //coming soon - [部署推理服务](docs/zh/tutorials/deployment.md) ## 模型列表 目前，MindCV支持以下模型。 -
+
支持模型 * Big Transfer ResNetV2 (BiT) - https://arxiv.org/abs/1912.11370 @@ -286,7 +250,7 @@ python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download ## 支持算法 -
+
支持算法列表 * 数据增强 @@ -295,12 +259,12 @@ python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download * [Repeated Augmentation](https://openaccess.thecvf.com/content_CVPR_2020/papers/Hoffer_Augment_Your_Batch_Improving_Generalization_Through_Instance_Repetition_CVPR_2020_paper.pdf) * RandErasing (Cutout) * CutMix - * Mixup + * MixUp * RandomResizeCrop * Color Jitter, Flip, etc * 优化器 * Adam - * Adamw + * AdamW * [Lion](https://arxiv.org/abs/2302.06675) * Adan (experimental) * AdaGrad @@ -329,9 +293,22 @@ python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download
-## 日志 +## 更新 + +- 2023/5/30 +1. 新模型: + - [VGG](configs/vgg)混合精度(O2)版本 + - [GhostNet](configs/ghostnet) + - [MobileNetV2](configs/mobilenetv2) 和 [MobileNetV3](configs/mobilenetv3)混合精度(O3)版本 + - [RegNet](configs/regnet)的(x,y)_(200,400,600,800)mf版本 + - [RepVGG](configs/repvgg)的b1g2, b1g4 & b2g4版本 + - [MnasNet](configs/mnasnet)的0.5版本 + - [PVTv2](configs/pvt_v2)的b3 & b4版本 +2. 新特性: + - 3-Augment, Augmix, TrivialAugmentWide +3. 错误修复: + - ViT 池化模式 -### 更新 - 2023/04/28 1. 增添了一些新模型，列出如下： - [VGG](configs/vgg) @@ -380,49 +357,45 @@ python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download 2. 新增4个模型的预训练权重及其策略： googlenet, inception_v3, inception_v4, xception - 2022/12/09 - 1. 支持在所有学习率策略中添加学习率预热操作，除cosine decay策略外。 2. 支持`Repeated Augmenation`操作，可以通过`--aug_repeats`对其进行设置，设置值应大于1(通常为3或4)。 3. 支持EMA。 4. 通过支持mixup和cutmix操作进一步优化BCE损失函数。 - 2022/11/21 - 1. 支持模型损失和正确率的可视化。 2. 支持伦次维度的cosine decay策略的学习率预热操作（之前仅支持步维度）。 - 2022/11/09 - 1. 支持2个ViT预训练模型。 2. 支持RandAugment augmentation操作。 3. 提高了CutMix操作的可用性，CutMix和Mixup目前可以一起使用。 4. 解决了学习率画图的bug。 - 2022/10/12 - 1. BCE和CE损失函数目前都支持class-weight config操作、label smoothing操作、auxilary logit input操作（适用于类似Inception模型）。 - 2022/09/13 - 1. 支持Adan优化器(试用版)。 -### 贡献方式 +## 贡献方式 欢迎开发者用户提issue或提交代码PR，或贡献更多的算法和模型，一起让MindCV变得更好。 -有关贡献指南，请参阅[CONTRIBUTING.md](CONTRIBUTING.md)。请遵循[模型编写指南](docs/zh/how_to_guides/write_a_new_model.md)所规定的规则来贡献模型接口：) +有关贡献指南，请参阅[CONTRIBUTING.md](CONTRIBUTING.md)。 +请遵循[模型编写指南](docs/zh/how_to_guides/write_a_new_model.md)所规定的规则来贡献模型接口：) -### 许可证 +## 许可证 本项目遵循[Apache License 2.0](LICENSE.md)开源协议。 -### 致谢 +## 致谢 MindCV是由MindSpore团队、西安电子科技大学、西安交通大学联合开发的开源项目。 衷心感谢所有参与的研究人员和开发人员为这个项目所付出的努力。 十分感谢 [OpenI](https://openi.pcl.ac.cn/) 平台所提供的算力资源。 -### 引用 +## 引用 如果你觉得MindCV对你的项目有帮助，请考虑引用： diff --git a/RELEASE.md b/RELEASE.md index 46f39cf6f..cb1069ae0 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,6 +1,97 @@ - # Release Note +- 2023/5/30 +1. New Models: + - AMP(O2) version of [VGG](configs/vgg) + - [GhostNet](configs/ghostnet) + - AMP(O3) version of [MobileNetV2](configs/mobilenetv2) and [MobileNetV3](configs/mobilenetv3) + - (x,y)_(200,400,600,800)mf of [RegNet](configs/regnet) + - b1g2, b1g4 & b2g4 of [RepVGG](configs/repvgg) + - 0.5 of [MnasNet](configs/mnasnet) + - b3 & b4 of [PVTv2](configs/pvt_v2) +2. New Features: + - 3-Augment, Augmix, TrivialAugmentWide +3. Bug Fixes: + - ViT pooling mode + +- 2023/04/28 +1. Add some new models, listed as following + - [VGG](configs/vgg) + - [DPN](configs/dpn) + - [ResNet v2](configs/resnetv2) + - [MnasNet](configs/mnasnet) + - [MixNet](configs/mixnet) + - [RepVGG](configs/repvgg) + - [ConvNeXt](configs/convnext) + - [Swin Transformer](configs/swintransformer) + - [EdgeNeXt](configs/edgenext) + - [CrossViT](configs/crossvit) + - [XCiT](configs/xcit) + - [CoAT](configs/coat) + - [PiT](configs/pit) + - [PVT v2](configs/pvt_v2) + - [MobileViT](configs/mobilevit) +2. Bug fix: + - Setting the same random seed for each rank + - Checking if options from yaml config exist in argument parser + - Initializing flag variable as `Tensor` in Optimizer `Adan` + +## 0.2.0 + +- 2023/03/25 +1. Update checkpoints for pretrained ResNet for better accuracy + - ResNet18 (from 70.09 to 70.31 @Top1 accuracy) + - ResNet34 (from 73.69 to 74.15 @Top1 accuracy) + - ResNet50 (from 76.64 to 76.69 @Top1 accuracy) + - ResNet101 (from 77.63 to 78.24 @Top1 accuracy) + - ResNet152 (from 78.63 to 78.72 @Top1 accuracy) +2. Rename checkpoint file name to follow naming rule ({model_scale-sha256sum.ckpt}) and update download URLs. + +- 2023/03/05 +1. Add Lion (EvoLved Sign Momentum) optimizer from paper https://arxiv.org/abs/2302.06675 + - To replace adamw with lion, LR is usually 3-10x smaller, and weight decay is usually 3-10x larger than adamw. +2. Add 6 new models with training recipes and pretrained weights for + - [HRNet](configs/hrnet) + - [SENet](configs/senet) + - [GoogLeNet](configs/googlenet) + - [Inception V3](configs/inception_v3) + - [Inception V4](configs/inception_v4) + - [Xception](configs/xception) +3. Support gradient clip +4. Arg name `use_ema` changed to **`ema`**, add `ema: True` in yaml to enable EMA. + +## 0.1.1 + +- 2023/01/10 +1. MindCV v0.1 released! It can be installed via PyPI `pip install mindcv` now. +2. Add training recipe and trained weights of googlenet, inception_v3, inception_v4, xception + +## 0.1.0 + +- 2022/12/09 +1. Support lr warmup for all lr scheduling algorithms besides cosine decay. +2. Add repeated augmentation, which can be enabled by setting `--aug_repeats` to be a value larger than 1 (typically, 3 or 4 is a common choice). +3. Add EMA. +4. Improve BCE loss to support mixup/cutmix. + +- 2022/11/21 +1. Add visualization for loss and acc curves +2. Support epochwise lr warmup cosine decay (previous is stepwise) + +- 2022/11/09 +1. Add 7 pretrained ViT models. +2. Add RandAugment augmentation. +3. Fix CutMix efficiency issue and CutMix and Mixup can be used together. +4. Fix lr plot and scheduling bug. + +- 2022/10/12 +1. Both BCE and CE loss now support class-weight config, label smoothing, and auxiliary logit input (for networks like inception). + +## 0.0.1-beta + +- 2022/09/13 +1. Add Adan optimizer (experimental) + ## MindSpore Computer Vision 0.0.1 ### Models diff --git a/docs/en/index.md b/docs/en/index.md index b8451e993..b5a7e2ab9 100644 --- a/docs/en/index.md +++ b/docs/en/index.md @@ -85,7 +85,7 @@ Below are a few code snippets for your taste.

- Classify the dowloaded image with a pretrained SoTA model: + Classify the downloaded image with a pretrained SoTA model: ```pycon >>> !python infer.py --model=swin_tiny --image_path='./dog.jpg' @@ -156,7 +156,7 @@ It is easy to train your model on a standard or customized dataset using `train. [Pynative mode with ms_function](https://www.mindspore.cn/tutorials/zh-CN/r1.8/advanced/pynative_graph/combine.html) is a mixed mode for comprising flexibility and efficiency in MindSpore. To apply pynative mode with ms_function for training, please run `train_with_func.py`, e.g., - ``` shell + ```shell python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download --epoch_size=10 ``` diff --git a/docs/zh/index.md b/docs/zh/index.md index 15ad16e21..de0f4a7a2 100644 --- a/docs/zh/index.md +++ b/docs/zh/index.md @@ -135,7 +135,6 @@ MindCV是一个基于 [MindSpore](https://www.mindspore.cn/) 开发的，致力 具体的参数配置和详细精度性能汇总请见[`configs`](https://github.com/mindspore-lab/mindcv/tree/main/configs)文件夹。 您可以便捷将这些训练策略用于您的模型训练中以提高性能（复用或修改相应的yaml文件即可） - - 在ModelArts/OpenI平台上训练 在[ModelArts](https://www.huaweicloud.com/intl/en-us/product/modelarts.html)或[OpenI](https://openi.pcl.ac.cn/)云平台上进行训练，需要执行以下操作，： @@ -177,7 +176,7 @@ python validate.py --model=resnet50 --dataset=imagenet --data_dir=/path/to/data ```shell python train.py --model=resnet50 --dataset=cifar10 \ - --val_while_train --val_split=test --val_interval=1 + --val_while_train --val_split=test --val_interval=1 ``` 各轮次的训练损失和测试精度将保存在`{ckpt_save_dir}/results.log`中。 @@ -206,7 +205,7 @@ python validate.py --model=resnet50 --dataset=imagenet --data_dir=/path/to/data * [Repeated Augmentation](https://openaccess.thecvf.com/content_CVPR_2020/papers/Hoffer_Augment_Your_Batch_Improving_Generalization_Through_Instance_Repetition_CVPR_2020_paper.pdf) * RandErasing (Cutout) * CutMix - * Mixup + * MixUp * RandomResizeCrop * Color Jitter, Flip, etc * 优化器 diff --git a/tutorials/imagenet1000_clsidx_to_labels.txt b/examples/data/imagenet1000_clsidx_to_labels.txt similarity index 100% rename from tutorials/imagenet1000_clsidx_to_labels.txt rename to examples/data/imagenet1000_clsidx_to_labels.txt diff --git a/infer.py b/infer.py index 06889d736..af262fa81 100644 --- a/infer.py +++ b/infer.py @@ -47,7 +47,7 @@ def main(): logits = nn.Softmax()(network(ms.Tensor(img)))[0].asnumpy() preds = np.argsort(logits)[::-1][:5] probs = logits[preds] - with open("./tutorials/imagenet1000_clsidx_to_labels.txt", encoding="utf-8") as f: + with open("./examples/data/imagenet1000_clsidx_to_labels.txt", encoding="utf-8") as f: idx2label = ast.literal_eval(f.read()) # print(f"Predict result of {args.image_path}:") cls_prob = {} diff --git a/setup.py b/setup.py index 3d1fac604..9783485f6 100644 --- a/setup.py +++ b/setup.py @@ -1,19 +1,28 @@ #!/usr/bin/env python +from pathlib import Path + from setuptools import find_packages, setup +# read the contents of README file +this_directory = Path(__file__).parent +long_description = (this_directory / "README.md").read_text() + +# read the `__version__` global variable in `version.py` exec(open("mindcv/version.py").read()) setup( name="mindcv", - author="MindSpore Ecosystem", - author_email="mindspore-ecosystem@example.com", + author="MindSpore Lab", + author_email="mindspore-lab@example.com", url="https://github.com/mindspore-lab/mindcv", project_urls={ "Sources": "https://github.com/mindspore-lab/mindcv", "Issue Tracker": "https://github.com/mindspore-lab/mindcv/issues", }, description="A toolbox of vision models and algorithms based on MindSpore.", + long_description=long_description, + long_description_content_type="text/markdown", license="Apache Software License 2.0", include_package_data=True, packages=find_packages(include=["mindcv", "mindcv.*"]), diff --git a/tutorials/README.md b/tutorials/README.md deleted file mode 100644 index 322cfd37f..000000000 --- a/tutorials/README.md +++ /dev/null @@ -1 +0,0 @@ -This folder contains jupyter notebooks used in the tutorial. diff --git a/tutorials/data/test/dog/dog.jpg b/tutorials/data/test/dog/dog.jpg deleted file mode 100644 index 81c91b1d3..000000000 Binary files a/tutorials/data/test/dog/dog.jpg and /dev/null differ