-
@@ -34,45 +34,46 @@ English | [简体中文](README_CN.md)
Introduction
PaddleSeg is an end-to-end high-efficent development toolkit for image segmentation based on PaddlePaddle, which helps both developers and researchers in the whole process of designing segmentation models, training models, optimizing performance and inference speed, and deploying models. A lot of well-trained models and various real-world applications in both industry and academia help users conveniently build hands-on experiences in image segmentation.
-* #### Four segmentation areas: semantic segmentation, interactive segmentation, panoptic segmentation and image matting.
-
-
+
----------------
-
- * #### Various applications in autonomous driving, medical segmentation, remote sensing, quality inspection, and other scenarios.
-
+
-
+
----------------
+##
+
Features
-## Features
+* **High-Performance Model**: Following the state of the art segmentation methods and use the high-performance backbone trained by semi-supervised label knowledge distillation scheme ([SSLD]((https://paddleclas.readthedocs.io/zh_CN/latest/advanced_tutorials/distillation/distillation.html#ssld))), we provide 40+ models and 140+ high-quality pre-training models, which are better than other open-source implementations.
-* 
**High-Performance Model**: Based on the high-performance backbone trained by semi-supervised label knowledge distillation scheme ([SSLD]((https://paddleclas.readthedocs.io/zh_CN/latest/advanced_tutorials/distillation/distillation.html#ssld))), combined with the state of the art segmentation technology, we provide 80+ high-quality pre-training models, which are better than other open-source implementations.
+* **High Efficiency**: PaddleSeg provides multi-process asynchronous I/O, multi-card parallel training, evaluation, and other acceleration strategies, combined with the memory optimization function of the PaddlePaddle, which can greatly reduce the training overhead of the segmentation model, all this allowing developers to lower cost and more efficiently train image segmentation model.
-* 
**Modular Design**: PaddleSeg supports 40+ mainstream *segmentation networks*, developers can start based on actual application scenarios and assemble diversified training configurations combined with modular design of *data enhancement strategies*, *backbone networks*, *loss functions* and other different components to meet different performance and accuracy requirements.
+* **Modular Design**: We desigin PaddleSeg with the modular design philosophy. Therefore, based on actual application scenarios, developers can assemble diversified training configurations with *data enhancement strategies*, *segmentation models*, *backbone networks*, *loss functions* and other different components to meet different performance and accuracy requirements.
-* 
**High Efficiency**: PaddleSeg provides multi-process asynchronous I/O, multi-card parallel training, evaluation, and other acceleration strategies, combined with the memory optimization function of the PaddlePaddle, which can greatly reduce the training overhead of the segmentation model, all this allowing developers to lower cost and more efficiently train image segmentation model.
+* **Complete Flow**: PaddleSeg support image labeling, model designing, model training, model compression and model deployment. With the help of PaddleSeg, developers can easily finish all taskes.
-
+
----
+##
+
Community
+
+* If you have any questions, suggestions and feature requests, please create an issues in [GitHub Issues](https://github.com/PaddlePaddle/PaddleSeg/issues).
+* Welcome to scan the following QR code and join paddleseg wechat group to communicate with us.
+
+
+
+
-## Overview
+
+## Overview
| - Projects + Special Case | |||
+
Semantic Segmentation+
Interactive Segmentation+
Image Matting+
Panoptic Segmentation+
|
- Backbones + Backbone
+ + Loss
+ Metrics
|
- Datasets + Dataset
-
Data Augmentation+
|
- Interactive Segmentation + Human Segmentation
-
-
-
-
+ 3D Medical Segmentation
-
-
Cityscapes SOTA Model+
CVPR Champion Model+
Domain Adaptation+
|
Industrial Segmentation Models
-
-
-
+
-
+
+
+
+High Accuracy Semantic Segmentation Models
+ +#### These models have good performance and costly inference time, so they are designed for GPU and Jetson devices. + +| Model | Backbone | Cityscapes mIoU(%) | V100 TRT Inference Speed(FPS) | Config File | +|:-------- |:--------:|:---------------------:|:-------------------------------:|:------------:| +| FCN | HRNet_W18 | 78.97 | 24.43 | [yml](./configs/fcn/) | +| FCN | HRNet_W48 | 80.70 | 10.16 | [yml](./configs/fcn/) | +| DeepLabV3 | ResNet50_OS8 | 79.90 | 4.56 | [yml](./configs/deeplabv3/) | +| DeepLabV3 | ResNet101_OS8 | 80.85 | 3.2 | [yml](./configs/deeplabv3/) | +| DeepLabV3 | ResNet50_OS8 | 80.36 | 6.58 | [yml](./configs/deeplabv3p/) | +| DeepLabV3 | ResNet101_OS8 | 81.10 | *3.94* | [yml](./configs/deeplabv3p/) | +| **OCRNet** | HRNet_w18 | 80.67 | 13.26 | [yml](./configs/ocrnet/) | +| OCRNet | HRNet_w48 | 82.15 | 6.17 | [yml](./configs/ocrnet/) | +| CCNet | ResNet101_OS8 | 80.95 | 3.24 | [yml](./configs/ccnet/) | + +Note that: +* Test the inference speed on Nvidia GPU V100: use PaddleInference Python API, enable TensorRT, the data type is FP32, the dimension of input is 1x3x1024x2048. + +
+
+
-* [Installation Guide](./docs/install.md)
-* [Quick Start](./docs/whole_process.md)
+
+Lightweight Semantic Segmentation Models
+ +#### The segmentation accuracy and inference speed of these models are medium. They can be deployed on GPU, X86 CPU and ARM CPU. +| Model | Backbone | Cityscapes mIoU(%) | V100 TRT Inference Speed(FPS) | Snapdragon 855 Inference Speed(FPS) | Config File | +|:-------- |:--------:|:---------------------:|:-------------------------------:|:-----------------:|:--------:| +| **PP-LiteSeg** | STDC1 | 77.04 | 69.82 | 17.22 | [yml](./configs/pp_liteseg/) | +| **PP-LiteSeg** | STDC2 | 79.04 | 54.53 | 11.75 | [yml](./configs/pp_liteseg/) | +| BiSeNetV1 | - | 75.19 | 14.67 | 1.53 |[yml](./configs/bisenetv1/) | +| BiSeNetV2 | - | 73.19 | 61.83 | 13.67 |[yml](./configs/bisenet/) | +| STDCSeg | STDC1 | 74.74 | 62.24 | 14.51 |[yml](./configs/stdcseg/) | +| STDCSeg | STDC2 | 77.60 | 51.15 | 10.95 |[yml](./configs/stdcseg/) | +| DDRNet_23 | - | 79.85 | 42.64 | 7.68 |[yml](./configs/ddrnet/) | +| HarDNet | - | 79.03 | 30.3 | 5.44 |[yml](./configs/hardnet/) | +| SFNet | ResNet18_OS8 | 78.72 | *10.72* | - | [yml](./configs/sfnet/) | +Note that: +* Test the inference speed on Nvidia GPU V100: use PaddleInference Python API, enable TensorRT, the data type is FP32, the dimension of input is 1x3x1024x2048. +* Test the inference speed on Snapdragon 855: use PaddleLite CPP API, 1 thread, the dimension of input is 1x3x256x256. -## Tutorials
+
+
+
+
+## Super Lightweight Semantic Segmentation Models
+ +#### These super lightweight semantic segmentation models are designed for X86 CPU and ARM CPU. + +| Model | Backbone | Cityscapes mIoU(%) | V100 TRT Inference Speed(FPS) | Snapdragon 855 Inference Speed(FPS) | Config File | +|:-------- |:--------:|:---------------------:|:-------------------------------:|:-----------------------------------:|:-----------:| +| MobileSeg | MobileNetV2 | 73.94 | 67.57 | 27.01 | [yml](./configs/mobileseg/) | +| **MobileSeg** | MobileNetV3 | 73.47 | 67.39 | 32.90 | [yml](./configs/mobileseg/) | +| MobileSeg | Lite_HRNet_18 | 70.75 | *10.5* | 13.05 | [yml](./configs/mobileseg/) | +| MobileSeg | ShuffleNetV2_x1_0 | 69.46 | *37.09* | 39.61 | [yml](./configs/mobileseg/) | +| MobileSeg | GhostNet_x1_0 | 71.88 | *35.58* | 38.74 | [yml](./configs/mobileseg/) | + +Note that: +* Test the inference speed on Nvidia GPU V100: use PaddleInference Python API, enable TensorRT, the data type is FP32, the dimension of input is 1x3x1024x2048. +* Test the inference speed on Snapdragon 855: use PaddleLite CPP API, 1 thread, the dimension of input is 1x3x256x256. + + Tutorials
+
+### Basic Tutorials
+
+* [Installation](./docs/install.md)
+* [Quick Start](./docs/quick_start.md)
* Data Preparation
* [Annotated Data Preparation](./docs/data/marker/marker.md)
* [Annotation Tutorial](./docs/data/transform/transform.md)
* [Custom Dataset](./docs/data/custom/data_prepare.md)
+* [Config Preparation](./docs/prepare_cfg.md)
* [Model Training](/docs/train/train.md)
* [Model Evaluation](./docs/evaluation/evaluate/evaluate.md)
@@ -300,6 +383,8 @@ The relationship between mIoU and FLOPs of representative architectures and back
* [Paddle JS](./docs/deployment/web/web.md)
* [Benchmark](./docs/deployment/inference/infer_benchmark.md)
+### Advanced Tutorials
+
* Model Compression
* [Quantization](./docs/slim/quant/quant.md)
* [Distillation](./docs/slim/distill/distill.md)
@@ -332,15 +417,6 @@ The relationship between mIoU and FLOPs of representative architectures and back
* [CVPR Champion Solution](./contrib/AutoNUE)
* [Domain Adaptation](./contrib/DomainAdaptation)
-
-## Community
-
-* If you have any problem or suggestion on PaddleSeg, please send us issues through [GitHub Issues](https://github.com/PaddlePaddle/PaddleSeg/issues).
-* Welcome to Join PaddleSeg WeChat Group
-
-
-
-
## License
PaddleSeg is released under the [Apache 2.0 license](LICENSE).
diff --git a/README_CN.md b/README_CN.md
index c1c92d5829..a3b7e92c99 100644
--- a/README_CN.md
+++ b/README_CN.md
@@ -6,7 +6,7 @@
-
-**飞桨高性能图像分割开发套件，端到端地完成从训练到部署的全流程图像分割应用。**
+**强大易用的飞桨图像分割开发套件，端到端完成从训练到部署的全流程图像分割应用。**
[](https://travis-ci.org/PaddlePaddle/PaddleSeg)
[](LICENSE)
@@ -17,7 +17,7 @@
-## 最新动态 
+## 最新动态
* [2022-04-20] :fire: PaddleSeg 2.5版本发布！详细发版信息请参考[Release Note](https://github.com/PaddlePaddle/PaddleSeg/releases)。
* 发布超轻量级语义分割模型[PP-LiteSeg](./configs/pp_liteseg)以及[技术报告](https://arxiv.org/pdf/2204.02681.pdf)，实现精度和速度的最佳平衡。
@@ -30,127 +30,147 @@
-## 简介
-PaddleSeg是基于飞桨PaddlePaddle开发的端到端图像分割开发套件，涵盖了**高精度**和**轻量级**等不同方向的大量高质量分割模型。通过模块化的设计，提供了**配置化驱动**和**API调用**两种应用方式，帮助开发者更便捷地完成从训练到部署的全流程图像分割应用。
+## 简介
-* #### 提供语义分割、交互式分割、全景分割、Matting四大图像分割能力。
+**PaddleSeg**是基于飞桨PaddlePaddle的端到端图像分割套件，内置**40+模型算法**及**140+预训练模型**，支持**配置化驱动**和**API调用**开发方式，打通数据标注、模型开发、训练、压缩、部署的**全流程**，提供**语义分割、交互式分割、Matting、全景分割**四大分割能力，助力算法在工业、自动驾驶、医疗、娱乐等场景落地应用。
-
+
-
-
-
-
----------------
-
- * #### 广泛应用在自动驾驶、医疗、质检、巡检、娱乐等场景。
-
+
-
+
-----------------
-## 特性
+
+## 特性
+* **高精度**：跟踪学术界的前沿分割技术，结合半监督标签知识蒸馏方案([SSLD](https://paddleclas.readthedocs.io/zh_CN/latest/advanced_tutorials/distillation/distillation.html#ssld))训练的骨干网络，提供40+主流分割网络、140+的高质量预训练模型，效果优于其他开源实现。
-* **高精度模型**：基于半监督标签知识蒸馏方案([SSLD](https://paddleclas.readthedocs.io/zh_CN/latest/advanced_tutorials/distillation/distillation.html#ssld))训练得到高精度骨干网络，结合前沿的分割技术，提供了80+的高质量预训练模型，效果优于其他开源实现。
+* **高性能**：使用多进程异步I/O、多卡并行训练、评估等加速策略，结合飞桨核心框架的显存优化功能，大幅度减少分割模型的训练开销，让开发者更低成本、更高效地完成图像分割训练。
-* **模块化设计**：支持40+主流分割网络 ，结合模块化设计的数据增强策略 、骨干网络、损失函数等不同组件，开发者可以基于实际应用场景出发，组装多样化的训练配置，满足不同性能和精度的要求。
+* **模块化**：源于模块化设计思想，解耦数据准备、分割模型、骨干网络、损失函数等不同组件，开发者可以基于实际应用场景出发，组装多样化的配置，满足不同性能和精度的要求。
-* **高性能**：支持多进程异步I/O、多卡并行训练、评估等加速策略，结合飞桨核心框架的显存优化功能，可大幅度减少分割模型的训练开销，让开发者更低成本、更高效地完成图像分割训练。
+* **全流程**：打通数据标注、模型开发、模型训练、模型压缩、模型部署全流程，经过业务落地的验证，让开发者完成一站式开发工作。
-
+
-----------
+##
+
技术交流
-## 技术交流
+* 如果大家有使用问题、产品建议、功能需求, 可以通过[GitHub Issues](https://github.com/PaddlePaddle/PaddleSeg/issues)提issues。
+* 欢迎大家扫码加入PaddleSeg微信群，和小伙伴们一起交流学习。
-* 如果你发现任何PaddleSeg存在的问题或者是建议, 欢迎通过[GitHub Issues](https://github.com/PaddlePaddle/PaddleSeg/issues)给我们提issues。
-* 欢迎加入PaddleSeg 微信群
+## 产品矩阵
| - 分割模型 + 模型 | - 分割组件 + 组件 | - 实践案例 + 特色案例 | |
+
语义分割模型+
交互式分割模型+
图像抠图模型+
全景分割+
|
- 骨干网络 + 骨干网络
+ + 损失函数
+ 评估指标
|
- 支持数据集 + 支持数据集
-
数据增强+
|
- 交互式分割 + 人像分割模型
-
-
-
-
+ 3D医疗分割模型
-
-
Cityscapes打榜模型+
CVPR冠军模型+
领域自适应+
|
+## 产业级分割模型库
-模型结构和骨干网络的代表模型在Cityscapes数据集mIoU和FLOPs对比图。请参见[Model Zoo Overview](./docs/model_zoo_overview_cn.md)了解更多模型信息以及对比图。
+
+
+
+
+高精度语义分割模型
-
-
-
+#### 高精度模型，分割mIoU高、推理算量大，适合部署在服务器端GPU和Jetson等设备。
+
+| 模型名称 | 骨干网络 | Cityscapes精度mIoU(%) | V100 TRT推理速度(FPS) | 配置文件 |
+|:-------- |:--------:|:---------------------:|:---------------------:|:--------:|
+| FCN | HRNet_W18 | 78.97 | 24.43 | [yml](./configs/fcn/) |
+| FCN | HRNet_W48 | 80.70 | 10.16 | [yml](./configs/fcn/) |
+| DeepLabV3 | ResNet50_OS8 | 79.90 | 4.56 | [yml](./configs/deeplabv3/) |
+| DeepLabV3 | ResNet101_OS8 | 80.85 | 3.2 | [yml](./configs/deeplabv3/) |
+| DeepLabV3 | ResNet50_OS8 | 80.36 | 6.58 | [yml](./configs/deeplabv3p/) |
+| DeepLabV3 | ResNet101_OS8 | 81.10 | *3.94* | [yml](./configs/deeplabv3p/) |
+| **OCRNet** | HRNet_w18 | 80.67 | 13.26 | [yml](./configs/ocrnet/) |
+| OCRNet | HRNet_w48 | 82.15 | 6.17 | [yml](./configs/ocrnet/) |
+| CCNet | ResNet101_OS8 | 80.95 | 3.24 | [yml](./configs/ccnet/) |
+
+
+测试条件：
+* V100上测速条件：针对Nvidia GPU V100，使用PaddleInference预测库的Python API，开启TensorRT加速，数据类型是FP32，输入图像维度是1x3x1024x2048。
+
+
-
+
+
+
+轻量级语义分割模型
+ +#### 轻量级模型，分割mIoU中等、推理算量中等，可以部署在服务器端GPU、服务器端X86 CPU和移动端ARM CPU。 + +| 模型名称 | 骨干网络 | Cityscapes精度mIoU(%) | V100 TRT推理速度(FPS) | 骁龙855推理速度(FPS) | 配置文件 | +|:-------- |:--------:|:---------------------:|:---------------------:|:-----------------:|:--------:| +| **PP-LiteSeg** | STDC1 | 77.04 | 69.82 | 17.22 | [yml](./configs/pp_liteseg/) | +| **PP-LiteSeg** | STDC2 | 79.04 | 54.53 | 11.75 | [yml](./configs/pp_liteseg/) | +| BiSeNetV1 | - | 75.19 | 14.67 | 1.53 |[yml](./configs/bisenetv1/) | +| BiSeNetV2 | - | 73.19 | 61.83 | 13.67 |[yml](./configs/bisenet/) | +| STDCSeg | STDC1 | 74.74 | 62.24 | 14.51 |[yml](./configs/stdcseg/) | +| STDCSeg | STDC2 | 77.60 | 51.15 | 10.95 |[yml](./configs/stdcseg/) | +| DDRNet_23 | - | 79.85 | 42.64 | 7.68 |[yml](./configs/ddrnet/) | +| HarDNet | - | 79.03 | 30.3 | 5.44 |[yml](./configs/hardnet/) | +| SFNet | ResNet18_OS8 | 78.72 | *10.72* | - | [yml](./configs/sfnet/) | + +测试条件： +* V100上测速条件：针对Nvidia GPU V100，使用PaddleInference预测库的Python API，开启TensorRT加速，数据类型是FP32，输入图像维度是1x3x1024x2048。 +* 骁龙855上测速条件：针对小米9手机，使用PaddleLite预测库的CPP API，ARMV8编译，单线程，输入图像维度是1x3x256x256。 + +
+
+## 超轻量级语义分割模型
+ +#### 超轻量级模型，分割mIoU一般、推理算量低，适合部署在服务器端X86 CPU和移动端ARM CPU。 + +| 模型名称 | 骨干网络 | Cityscapes精度mIoU(%) | V100 TRT推理速度(FPS) | 骁龙855推理速度(FPS)| 配置文件 | +|:-------- |:--------:|:---------------------:|:---------------------:|:-----------------:|:--------:| +| MobileSeg | MobileNetV2 | 73.94 | 67.57 | 27.01 | [yml](./configs/mobileseg/) | +| **MobileSeg** | MobileNetV3 | 73.47 | 67.39 | 32.90 | [yml](./configs/mobileseg/) | +| MobileSeg | Lite_HRNet_18 | 70.75 | *10.5* | 13.05 | [yml](./configs/mobileseg/) | +| MobileSeg | ShuffleNetV2_x1_0 | 69.46 | *37.09* | 39.61 | [yml](./configs/mobileseg/) | +| MobileSeg | GhostNet_x1_0 | 71.88 | *35.58* | 38.74 | [yml](./configs/mobileseg/) | + +测试条件： +* V100上测速条件：针对Nvidia GPU V100，使用PaddleInference预测库的Python API，开启TensorRT加速，数据类型是FP32，输入图像维度是1x3x1024x2048。 +* 骁龙855上测速条件：针对小米9手机，使用PaddleLite预测库的CPP API，ARMV8编译，单线程，输入图像维度是1x3x256x256。 + + 使用教程
-## 使用教程
+### 基础教程
-* [安装](./docs/install_cn.md)
-* [全流程跑通PaddleSeg](./docs/whole_process_cn.md)
-* 准备数据集
+* [安装说明](./docs/install_cn.md)
+* [快速体验](./docs/quick_start_cn.md)
+* 准备数据
* [标注数据的准备](./docs/data/marker/marker_cn.md)
* [数据标注教程](./docs/data/transform/transform_cn.md)
* [自定义数据集](./docs/data/custom/data_prepare_cn.md)
+* [准备配置文件](./docs/prepare_cfg_cn.md)
* [模型训练](/docs/train/train_cn.md)
* [模型评估](./docs/evaluation/evaluate/evaluate_cn.md)
-* [预测与可视化](./docs/predict/predict_cn.md)
+* [模型预测](./docs/predict/predict_cn.md)
* 模型导出
* [导出预测模型](./docs/model_export_cn.md)
@@ -298,6 +371,7 @@ PaddleSeg是基于飞桨PaddlePaddle开发的端到端图像分割开发套件
* [Paddle Serving部署](./docs/deployment/serving/serving.md)
* [Paddle JS部署](./docs/deployment/web/web_cn.md)
* [推理Benchmark](./docs/deployment/inference/infer_benchmark_cn.md)
+### 高阶教程
* 模型压缩
* [量化](./docs/slim/quant/quant_cn.md)
@@ -321,7 +395,7 @@ PaddleSeg是基于飞桨PaddlePaddle开发的端到端图像分割开发套件
* [技术交流](#技术交流)
* [常见问题汇总](./docs/faq/faq/faq_cn.md)
-## 实践案例 
+## 实践案例
- [交互式分割](./EISeg)
- [图像抠图](./Matting)
@@ -355,7 +429,7 @@ PaddleSeg是基于飞桨PaddlePaddle开发的端到端图像分割开发套件
- 非常感谢[ETTR123](https://github.com/ETTR123)(张恺) 贡献 ENet，PFPNNet。
-## 学术引用 
+## 学术引用
如果我们的项目在学术上帮助到你，请考虑以下引用：
diff --git a/docs/images/chat.png b/docs/images/chat.png
index cc6a034aac..96841931a8 100644
Binary files a/docs/images/chat.png and b/docs/images/chat.png differ
diff --git a/docs/images/seg_news_icon.png b/docs/images/seg_news_icon.png
index 30ec26f5fc..3731408266 100644
Binary files a/docs/images/seg_news_icon.png and b/docs/images/seg_news_icon.png differ
diff --git a/docs/images/teach.png b/docs/images/teach.png
index f57bb4b2fd..c38a4b9805 100644
Binary files a/docs/images/teach.png and b/docs/images/teach.png differ