[Feature] Add ESPNetV1 #1625

simuler · 2021-12-14T03:19:06Z

添加ESPNet模型（Dynamic Multi-scale Filters for Semantic Segmentation）

添加内容：
configs/espnetv1/espnetv1_cityscapes_1024x512_120k.yml
configs/espnetv1/README.md
paddleseg/models/espnetv1.py
paddleseg/models/init.py

shiyutang

Please read our style markdown before PR. This PR needs more adjustment than my comments.
Style markdown link: https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/pr/pr/style_cn.md#paddleseg%E6%A8%A1%E5%9E%8B%E5%BC%80%E5%8F%91%E8%A7%84%E8%8C%83

shiyutang · 2021-12-14T06:27:34Z

configs/espnetv1/README.md

+
+
+## Reference
+[1] Mehta S ,  Rastegari M ,  Caspi A , et al. ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation[J]. Springer, Cham, 2018.


Plz change this quote to Chicago format, which includes their full name.
you can refer to this: https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/pr/pr/style_cn.md#%E4%B8%89%E6%96%B0%E5%A2%9E-pr-checklist

shiyutang · 2021-12-14T06:32:24Z

configs/espnetv1/README.md

+## Reference
+[1] Mehta S ,  Rastegari M ,  Caspi A , et al. ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation[J]. Springer, Cham, 2018.
+## Performance
+CityScapes


shiyutang · 2021-12-14T06:33:48Z

configs/espnetv1/README.md

+| Model|steps|opt|image_size|batch_size|dataset|memory|card|mIou|config|
+| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+|ESPNet|120k|adam|1024x512|4|CityScapes|32G|4|0.6365|[espnet_cityscapes_1024x512_120k.yml](configs/espnetv1/espnetv1_cityscapes_1024x512_120k.yml)|


Actually, you do not need to include so much information. Please refer to our style MD before you bring up your PR

shiyutang · 2021-12-14T06:34:34Z

paddleseg/models/espnetv1.py

+def SyncBatchNorm(*args, **kwargs):
+    """In cpu environment nn.SyncBatchNorm does not have kernel so use nn.BatchNorm2D instead"""
+    if paddle.get_device() == 'cpu' or os.environ.get('PADDLESEG_EXPORT_STAGE'):
+        return nn.BatchNorm2D(*args, **kwargs)
+    elif paddle.distributed.ParallelEnv().nranks == 1:
+        return nn.BatchNorm2D(*args, **kwargs)
+    else:
+        return nn.SyncBatchNorm(*args, **kwargs)


please make use of the same function we have in layers.

shiyutang · 2021-12-14T06:35:18Z

paddleseg/models/espnetv1.py

+        p1, p2, p3 = self.encoder(x) # shape [N, C, H/2, W/2]  [N, C, H/4, W/4]  [N, C, H/8, W/8] 
+        up_p3 = self.level3_up(p3) # [N, C, H/4, W/4]
+
+        combine = self.combine_l2_l3(paddle.concat([up_p3, p2], axis=1))
+        up_p2 = self.level2_up(combine)  # [N, C, H/2, W/2]
+
+        combine = self.out_proj(paddle.concat([up_p2, p1], axis=1))  # shape [N, C, H/2, W/2]


Remove this annotation please, here and elsewhere

paddleseg/models/espnetv1.py

shiyutang · 2021-12-17T08:43:30Z

configs/espnetv1/espnetv1_cityscapes_1024x512_120k.yml

+batch_size: 4
+iters: 120000
+
+


remove this plz

shiyutang · 2021-12-17T08:44:35Z

paddleseg/models/espnetv1.py

+if __name__ == '__main__':
+    model = ESPNetV1(19, 3, 2, 8)
+    paddle.summary(model, (4, 3, 256, 256))


remove test code plz

shiyutang · 2021-12-17T08:46:02Z

paddleseg/models/espnetv1.py

+class Conv(nn.Layer):
+    def __init__(self, in_channels, out_channels, kernel_size, stride=1):
+        super().__init__()
+        padding = int((kernel_size - 1) / 2)
+        self.conv = nn.Conv2D(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias_attr=False)
+
+    def forward(self, x):
+        x = self.conv(x)
+        return x
+
+
+class ConvDilated(nn.Layer):
+    def __init__(self, in_channels, out_channels, kernel_size, stride=1, dilation=1):
+        super().__init__()
+        padding = int((kernel_size - 1) / 2) * dilation
+        self.conv = nn.Conv2D(in_channels, out_channels, kernel_size, stride=stride, padding=padding, dilation=dilation, bias_attr=False)
+
+    def forward(self, x):
+        x = self.conv(x)
+        return x


It seems these two function is not that necessary

shiyutang · 2021-12-17T08:46:32Z

paddleseg/models/espnetv1.py

+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+import os


This should be moved up

shiyutang · 2021-12-17T08:47:11Z

paddleseg/models/espnetv1.py

+class ConvBNPReLU(nn.Layer):
+    def __init__(self, in_channels, out_channels, kernel_size, stride=1):
+        super().__init__()
+        padding = int((kernel_size - 1) / 2)
+        self.conv = nn.Conv2D(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias_attr=False)
+        self.bn = layers.SyncBatchNorm(out_channels)
+        self.act = nn.PReLU(out_channels)
+
+    def forward(self, x):
+        x = self.conv(x)
+        x = self.bn(x)
+        x = self.act(x)
+        return x
+
+
+class BNPReLU(nn.Layer):
+    def __init__(self, channels):
+        super().__init__()
+        self.bn = layers.SyncBatchNorm(channels)
+        self.act = nn.PReLU(channels)
+
+    def forward(self, x):
+        x = self.bn(x)
+        x = self.act(x)
+        return x
+
+
+class ConvBN(nn.Layer):
+    def __init__(self, in_channels, out_channels, kernel_size, stride=1):
+        super().__init__()
+        padding = int((kernel_size - 1) / 2)
+        self.conv = nn.Conv2D(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias_attr=False)
+        self.bn = layers.SyncBatchNorm(out_channels)
+
+    def forward(self, x):
+        x = self.conv(x)
+        x = self.bn(x)
+        return x


please checkout layers package for these function

1、The ConvBNPReLU of layers package cannot modified the stride
`class ConvBNPReLU(nn.Layer):
def init(self,
in_channels,
out_channels,
kernel_size,
padding='same',
**kwargs):
super().init()

self._conv = nn.Conv2D(in_channels, out_channels, kernel_size, padding=padding, **kwargs) if 'data_format' in kwargs: data_format = kwargs['data_format'] else: data_format = 'NCHW' self._batch_norm = SyncBatchNorm(out_channels, data_format=data_format) self._prelu = layers.Activation("prelu") def forward(self, x): x = self._conv(x) x = self._batch_norm(x) x = self._prelu(x) return x`

2、no BNPReLU in layers package

3、 I checked the code again, ConvBN is not used

shiyutang · 2021-12-20T03:39:02Z

configs/espnetv1/README.md

+
+### Cityscapes
+
+| Model | Backbone | Resolution | Training Iters | mIoU | Links |


please add the result of mIOU(+flip) and +ms & flip like this:

You can get the result through:

flip+ms python3 val.py --config .yml --model_path best_model/model.pdparams --num_workers 2 --aug_eval --flip_horizontal --scales 0.75 1.0 1.25 flip python3 val.py --config config.yml --model_path best_model/model.pdparams --num_workers 2 --aug_eval --flip_horizontal

README.md is updated. Will you train espnetv1 from the scratch? If you do, wo don't supply weight trained before.

we will not train it again, you can give the model's weight to me.

shiyutang · 2021-12-21T07:42:21Z

paddleseg/models/espnetv1.py

+class ConvBNPReLU(nn.Layer):
+    def __init__(self, in_channels, out_channels, kernel_size, stride=1):
+        super().__init__()
+        padding = int((kernel_size - 1) / 2)
+        self.conv = nn.Conv2D(in_channels, out_channels, kernel_size, stride=stride, padding=padding, bias_attr=False)
+        self.bn = layers.SyncBatchNorm(out_channels)
+        self.act = nn.PReLU(out_channels)
+
+    def forward(self, x):
+        x = self.conv(x)
+        x = self.bn(x)
+        x = self.act(x)
+        return x


Actually the models.layers.ConvBNPReLU can pass in kwargs to change the padding and stride of Conv2D.

[Feature]Add ESPNetV1

7321290

shiyutang requested changes Dec 14, 2021

View reviewed changes

simuler added 2 commits December 14, 2021 16:16

fixed

899ce3e

[Feature]Add ESPNetV1

88a50fd

michaelowenliu changed the title ~~[Feature]Add ESPNetV1~~ [Feature] Add ESPNetV1 Dec 16, 2021

shiyutang requested changes Dec 17, 2021

View reviewed changes

[Feature] Add ESPNetV1

f592378

shiyutang requested changes Dec 21, 2021

View reviewed changes

justld and others added 6 commits December 22, 2021 11:53

update readme.md

d8d9dfa

update espnetv1 and configs

24bd0c4

update format

842d0aa

update format

e909b2e

Merge branch 'develop' into espnetv1

1d0d12a

update format

43bb31d

simuler requested a review from shiyutang December 22, 2021 07:55

justld and others added 4 commits December 24, 2021 17:42

update miou value

c121264

Update README.md

1101e26

Update README.md

0fe3a37

trigger model check

f52ee0d

shiyutang approved these changes Dec 24, 2021

View reviewed changes

michaelowenliu approved these changes Dec 28, 2021

View reviewed changes

michaelowenliu merged commit bfca53f into PaddlePaddle:develop Dec 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Add ESPNetV1 #1625

[Feature] Add ESPNetV1 #1625

simuler commented Dec 14, 2021

shiyutang left a comment •

edited

Loading

shiyutang Dec 14, 2021

simuler Dec 14, 2021

shiyutang Dec 14, 2021

simuler Dec 14, 2021

shiyutang Dec 14, 2021

simuler Dec 14, 2021

shiyutang Dec 14, 2021

simuler Dec 14, 2021

shiyutang Dec 14, 2021

simuler Dec 14, 2021

shiyutang Dec 17, 2021

simuler Dec 20, 2021

shiyutang Dec 17, 2021

simuler Dec 20, 2021

shiyutang Dec 17, 2021

simuler Dec 20, 2021

shiyutang Dec 17, 2021

simuler Dec 20, 2021

shiyutang Dec 17, 2021

simuler Dec 20, 2021

shiyutang Dec 20, 2021

justld Dec 22, 2021

shiyutang Dec 23, 2021

justld Dec 24, 2021

shiyutang Dec 21, 2021

justld Dec 22, 2021



		## Reference
		[1] Mehta S , Rastegari M , Caspi A , et al. ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation[J]. Springer, Cham, 2018.


		### Cityscapes

		\| Model \| Backbone \| Resolution \| Training Iters \| mIoU \| Links \|

		batch_size: 4
		iters: 120000

[Feature] Add ESPNetV1 #1625

[Feature] Add ESPNetV1 #1625

Conversation

simuler commented Dec 14, 2021

shiyutang left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Cityscapes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shiyutang left a comment •

edited

Loading