Add PortraitNet. #1132

txyugood · 2021-07-08T01:24:59Z

Add portraitNet model.
Add RandomAffine in transform.py for data augmentation.
Add kl_loss and focal_loss for PortraintNet.
Modify train.py for PortraintNet.
Add EG1800 and Supervisely dataset.
Add stepdecay in config.py for training strategy.

LutaoChu · 2021-07-09T06:56:07Z

paddleseg/transforms/transforms.py

+
+
+@manager.TRANSFORMS.add_component
+class RandomAffine:


这个指缩放+旋转吧？是否可以通过已有的StepScaling+Rotation组合就好了，不用另外开发？

这个是平移、缩放和旋转。我之前试过通过StepScaling+Rotation但是效果完全不一样。所以按照论文实现了RandomAffine。

paddleseg/datasets/eg1800.py

paddleseg/datasets/supervisely.py

paddleseg/datasets/eg1800.py

paddleseg/models/portraitnet.py

LutaoChu · 2021-07-13T11:35:46Z

paddleseg/core/train.py

@@ -104,8 +109,10 @@ def train(model,
            optimizer)  # The return is Fleet object
        ddp_model = paddle.distributed.fleet.distributed_model(model)

-    batch_sampler = paddle.io.DistributedBatchSampler(
-        train_dataset, batch_size=batch_size, shuffle=True, drop_last=True)


yapf版本改为0.26.0，所有文件重新格式化一下吧

LutaoChu · 2021-07-13T11:36:32Z

paddleseg/datasets/eg1800.py

@@ -0,0 +1,119 @@
+# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.


所有文件license里的2020改成2021

LutaoChu · 2021-07-13T11:37:51Z

paddleseg/datasets/eg1800.py

+import paddleseg.transforms.functional as F
+import cv2
+import numpy as np
+import copy


所有文件的import顺序应该符合PEP8规范

LutaoChu · 2021-07-20T03:53:19Z

configs/portraitnet/portraitnet_eg8000_224x224_46k.yml

+  type: adam
+  weight_decay: 5.0e-4
+
+learning_rate:


最新用lr_scheduler，learning_rate要被废弃了

LutaoChu · 2021-07-20T03:54:31Z

paddleseg/datasets/eg1800.py

+    EG1800 dataset `http://xiaoyongshen.me/webpage_portrait/index.html`.
+
+    Args:
+        transforms (list): A list of image transformations.


缺少transforms2的注释

LutaoChu · 2021-07-20T06:23:44Z

paddleseg/datasets/supervisely.py

+            # add augmentation
+            std = np.array(self.std)[:, np.newaxis, np.newaxis]
+            mean = np.array(self.mean)[:, np.newaxis, np.newaxis]
+            im_aug = im * std + mean


这个地方逻辑有点别扭。最好是经过共同的transform后，出来两个分支transform，分别处理im和im_aug。
EG1800也是一样

LutaoChu · 2021-07-20T07:14:46Z

paddleseg/models/losses/focal_loss.py

+        self.gamma = gamma
+        self.ignore_index = ignore_index
+
+    def forward(self, inp, target):


inp, target改成logit，label吧

LutaoChu · 2021-07-20T07:19:08Z

paddleseg/models/backbones/mobilenetv2.py

+        The MobileNetV2 implementation based on PaddlePaddle.
+
+        The original article refers to
+        Song-Hai Zhanga, Xin Donga, Jia Lib, Ruilong Lia, Yong-Liang Yangc


Reference改成mobilenetv2

LutaoChu · 2021-07-20T07:56:33Z

paddleseg/models/losses/focal_loss.py

+    def forward(self, inp, target):
+        inp = paddle.reshape(inp, [inp.shape[0], inp.shape[1], -1])
+        inp = paddle.transpose(inp, [0, 2, 1])
+        inp = paddle.reshape(inp, [-1, 2])


应该是inp = paddle.reshape(inp, [-1, inp.shape[2])吧，用的时候不一定是二分类

加上注释 # N,H*W,C => N*H*W,C

LutaoChu · 2021-07-20T07:57:13Z

paddleseg/models/losses/focal_loss.py

+        self.ignore_index = ignore_index
+
+    def forward(self, inp, target):
+        inp = paddle.reshape(inp, [inp.shape[0], inp.shape[1], -1])


加上注释 # N,C,H,W => N,C,H*W

LutaoChu · 2021-07-20T07:57:36Z

paddleseg/models/losses/focal_loss.py

+
+    def forward(self, inp, target):
+        inp = paddle.reshape(inp, [inp.shape[0], inp.shape[1], -1])
+        inp = paddle.transpose(inp, [0, 2, 1])


加上注释 # N,C,H*W => N,H*W,C

LutaoChu · 2021-07-20T08:26:49Z

paddleseg/models/losses/kl_loss.py

+        self.kl_loss = nn.KLDivLoss(reduction="mean")
+        self.temperature = temperature
+
+    def forward(self, inp, target):


inp, target改成logit，label

LutaoChu · 2021-07-20T08:28:18Z

paddleseg/models/losses/kl_loss.py

+@manager.LOSSES.add_component
+class KLLoss(nn.Layer):
+    """
+    The Kullback-Leibler divergence Loss implement of Portrait Net.


去掉portraitnet
加上

Code referenced from: https://github.com/peterliht/knowledge-distillation-pytorch/blob/master/model/net.py Compute the knowledge-distillation (KD) loss given outputs, labels. "Hyperparameters": temperature and alpha

LutaoChu · 2021-07-20T09:52:55Z

paddleseg/models/losses/focal_loss.py

+            and does not contribute to the input gradient. Default ``255``.
+    """
+
+    def __init__(self, gamma=2.0, ignore_index=255):


为了能让focal loss更通用，加上edge_label=False参数，通过edge_label判断是不是用edge监督

LutaoChu · 2021-07-20T09:54:19Z

paddleseg/models/losses/focal_loss.py

+    def __init__(self, gamma=2.0, ignore_index=255):
+        super(FocalLoss, self).__init__()
+        self.gamma = gamma
+        self.ignore_index = ignore_index


加上self.edge_label = edge_label

LutaoChu · 2021-07-20T09:56:22Z

paddleseg/core/train.py

@@ -42,6 +42,11 @@ def loss_computation(logits_list, labels, losses, edges=None):
        # Whether to use edges as labels According to loss type.
        if loss_i.__class__.__name__ in ('BCELoss', ) and loss_i.edge_label:


改成 if loss_i.class.name in ('BCELoss', "FocalLoss") and loss_i.edge_label:

paddleseg/core/train.py

2.Add common transform for inputs of portrait net. 3.Standardize some variable names.

LutaoChu · 2021-07-22T08:04:50Z

paddleseg/models/portraitnet.py

+
+
+class PortraitNetHead(nn.Layer):
+    def __init__(self, num_classes, add_edge=False):


开放参数min_channel，channel_ratio吧

LutaoChu · 2021-07-22T08:05:12Z

paddleseg/models/portraitnet.py

+        pretrained (str, optional): The path or url of pretrained model. Default: None
+    """
+
+    def __init__(self, num_classes, backbone, add_edge=False, pretrained=None):


开放参数min_channel，channel_ratio吧

LutaoChu · 2021-07-22T08:07:17Z

paddleseg/models/backbones/mobilenetv2.py

+
+
+@manager.MODELS.add_component
+class MobileNetV2(nn.Layer):


提到文件最前头吧

Add PortraitNet.

ad9d853

LutaoChu requested changes Jul 9, 2021

View reviewed changes

txyugood added 2 commits July 12, 2021 10:02

Add some transforms for PortraitNet.

a30bd5e

Merge branch 'develop' into dev

404f030

LutaoChu previously approved these changes Jul 13, 2021

View reviewed changes

LutaoChu reviewed Jul 13, 2021

View reviewed changes

Format code of PortraitNet.

e3167de

txyugood dismissed LutaoChu’s stale review via e3167de July 13, 2021 12:15

remove unused code in eg1800.py.

992625f

txyugood force-pushed the dev branch from 0a64664 to 992625f Compare July 13, 2021 13:49

Update portraitnet_eg8000_224x224_46k.yml

f2f13ac

LutaoChu requested changes Jul 20, 2021

View reviewed changes

txyugood and others added 6 commits July 21, 2021 09:47

1.Modify the reference of mobilenet v2, kl loss and focal loss.

9affde7

2.Add common transform for inputs of portrait net. 3.Standardize some variable names.

Update README.md

98343e0

Update portraitnet_eg1800_224x224_46k.yml

d2a494b

Update supervisely.py

6e812ae

Update eg1800.py

4e2d5b5

Update README.md

dd42cf7

LutaoChu reviewed Jul 22, 2021

View reviewed changes

Update portraitnet.py and mobilenetv2.py.

bff5781

LutaoChu previously approved these changes Jul 22, 2021

View reviewed changes

Merge branch 'develop' into dev

044d3b5

LutaoChu dismissed their stale review via 044d3b5 July 30, 2021 03:34

michaelowenliu approved these changes Jul 30, 2021

View reviewed changes

LutaoChu closed this Aug 2, 2021

LutaoChu reopened this Aug 2, 2021

michaelowenliu merged commit 462da0d into PaddlePaddle:develop Aug 4, 2021

txyugood deleted the dev branch August 8, 2021 13:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PortraitNet. #1132

Add PortraitNet. #1132

txyugood commented Jul 8, 2021

LutaoChu Jul 9, 2021

txyugood Jul 9, 2021

LutaoChu Jul 13, 2021

LutaoChu Jul 13, 2021

LutaoChu Jul 13, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 20, 2021

LutaoChu Jul 22, 2021

LutaoChu Jul 22, 2021

LutaoChu Jul 22, 2021

		@@ -0,0 +1,119 @@
		# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.

		@@ -42,6 +42,11 @@ def loss_computation(logits_list, labels, losses, edges=None):
		# Whether to use edges as labels According to loss type.
		if loss_i.__class__.__name__ in ('BCELoss', ) and loss_i.edge_label:



		class PortraitNetHead(nn.Layer):
		def __init__(self, num_classes, add_edge=False):

Add PortraitNet. #1132

Add PortraitNet. #1132

Conversation

txyugood commented Jul 8, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment