[Feature] Add BasicVSR++ #451

ckkelvinchan · 2021-07-24T06:46:33Z

Paper: BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

3 champions + 1 runner-up in NTIRE 2021.

codecov · 2021-07-24T06:56:14Z

Codecov Report

Merging #451 (63938db) into master (6f6ebd6) will decrease coverage by 0.37%.
The diff coverage is 60.31%.

❗ Current head 63938db differs from pull request most recent head 91e697a. Consider uploading reports for the commit 91e697a to get more accurate results

@@            Coverage Diff             @@
##           master     #451      +/-   ##
==========================================
- Coverage   80.92%   80.54%   -0.38%     
==========================================
  Files         189      190       +1     
  Lines       10149    10338     +189     
  Branches     1495     1533      +38     
==========================================
+ Hits         8213     8327     +114     
- Misses       1721     1781      +60     
- Partials      215      230      +15

Flag	Coverage Δ
unittests	`80.52% <60.31%> (-0.38%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmedit/models/backbones/__init__.py	`100.00% <ø> (ø)`
...medit/models/backbones/sr_backbones/basicvsr_pp.py	`60.10% <60.10%> (ø)`
mmedit/models/backbones/sr_backbones/__init__.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6f6ebd6...91e697a. Read the comment docs.

mmedit/models/backbones/sr_backbones/basicvsr_pp.py

nbei · 2021-07-28T15:43:23Z

mmedit/models/backbones/sr_backbones/basicvsr_pp.py

+        o1, o2, mask = torch.chunk(out, 3, dim=1)
+
+        # offset
+        offset = 10 * torch.tanh(torch.cat((o1, o2), dim=1))


Why this number is 10?

You may give some comments on this number. Otherwise, you can make it user-defined but offer a default value of 10.

nbei · 2021-07-29T05:18:21Z

mmedit/models/backbones/sr_backbones/basicvsr_pp.py

+
+        return flows_forward, flows_backward
+
+    def propagate(self, feats, flows, module):


module_name may be better.

nbei · 2021-07-29T05:22:10Z

mmedit/models/backbones/sr_backbones/basicvsr_pp.py

+        """Compute the output image given the features.
+
+        Args:
+            lqs (tensor): Input LR images with shape (n, t, c, h, w).


What is lqs short for? You may specify the meaning of lqs.

nbei

LGTM

nbei · 2021-08-02T18:33:29Z

@innerlee may have a look.

innerlee · 2021-08-05T03:46:46Z

configs/restorers/basicvsr_plusplus/README.md

+| [basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bi](/configs/restorers/basicvsr/basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bi.py) |        31.0126/0.8804        |        **37.7864/0.9500**        |     **27.7882/0.8401**    |       33.1211/0.9270       |          33.8972/0.9195          |       23.6086/0.7033      | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bi_20210305-4ef437e2.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bi_20210305_141254.log.json) |
+| [basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bd](/configs/restorers/basicvsr/basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bd.py) |        29.2041/0.8528        |          34.7248/0.9351          |       26.4377/0.8074      |     **40.7216/0.9722**     |        **38.2054/0.9550**        |     **29.0400/0.8753**    | [model](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bd_20210305-ab315ab1.pth) \| [log](https://download.openmmlab.com/mmediting/restorers/basicvsr_plusplus/basicvsr_plusplus_c64n7_8x1_300k_vimeo90k_bd_20210305_140921.log.json) |
+
+## NTIRE 2021 checkpoints


wrap L29-39 to a <details> section

innerlee · 2021-08-05T07:34:57Z

tests/test_models/test_backbones/test_sr_backbones/test_basicvsr_plusplus.py

+def test_basicvsr_plusplus():
+    """Test BasicVSR++."""
+
+    # gpu (DCN can only be tested in GPUs)


Can DCN be turned off, and be tested on cpu?

Yes we can, but we need to create another argument to specify whether to skip the alignment.

Since this is one of the core parts of BasicVSR++, I am not sure whether adding this option is meaningful or not.

I just want to test it haha. It does not have to do with what basicvsr thinks

Or can you implement a cpu version of DCN?

Or can you implement a cpu version of DCN?

Not sure if I can do it in a short time haha. One workaround for now is to add the argument to skip alignment, and set it to False.

Or you can check the coverage report to see if more parts could be tested. 80% is a healthy line

innerlee · 2021-08-05T07:35:38Z

Dropping blow 80% is not good

innerlee · 2021-08-05T12:07:38Z

@ckkelvinchan Please add this method to the repo's README

* Add BasicVSR++ * Update * Add max_residue_magnitude as an argument * Add configuration files and README * Modify README * Modify README * Raise error about DCN if CUDA is not available * Skip alignment in CPU mode

ckkelvinchan added 2 commits July 24, 2021 14:42

Add BasicVSR++

ead9058

pull from origin

26b1979

nbei reviewed Jul 24, 2021

View reviewed changes

Update

afc2570

nbei reviewed Jul 28, 2021

View reviewed changes

nbei reviewed Jul 29, 2021

View reviewed changes

Add max_residue_magnitude as an argument

65f44df

nbei approved these changes Aug 2, 2021

View reviewed changes

nbei requested a review from innerlee August 2, 2021 18:33

ckkelvinchan added 2 commits August 4, 2021 21:23

Add configuration files and README

cd003af

Modify README

4c54d78

innerlee reviewed Aug 5, 2021

View reviewed changes

Modify README

364e789

innerlee approved these changes Aug 5, 2021

View reviewed changes

innerlee reviewed Aug 5, 2021

View reviewed changes

innerlee added the need test label Aug 5, 2021

ckkelvinchan added 2 commits August 5, 2021 16:30

Raise error about DCN if CUDA is not available

da9dc80

Skip alignment in CPU mode

91e697a

innerlee merged commit 41c061f into open-mmlab:master Aug 5, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Add BasicVSR++ #451

[Feature] Add BasicVSR++ #451

ckkelvinchan commented Jul 24, 2021

codecov bot commented Jul 24, 2021 •

edited

Loading

nbei Jul 28, 2021

nbei Jul 28, 2021

nbei Jul 29, 2021

nbei Jul 29, 2021

nbei left a comment

nbei commented Aug 2, 2021

innerlee Aug 5, 2021 •

edited

Loading

innerlee Aug 5, 2021

ckkelvinchan Aug 5, 2021

ckkelvinchan Aug 5, 2021

innerlee Aug 5, 2021

innerlee Aug 5, 2021

ckkelvinchan Aug 5, 2021

innerlee Aug 5, 2021

innerlee commented Aug 5, 2021

innerlee commented Aug 5, 2021


		return flows_forward, flows_backward

		def propagate(self, feats, flows, module):

[Feature] Add BasicVSR++ #451

[Feature] Add BasicVSR++ #451

Conversation

ckkelvinchan commented Jul 24, 2021

codecov bot commented Jul 24, 2021 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nbei left a comment

Choose a reason for hiding this comment

nbei commented Aug 2, 2021

innerlee Aug 5, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

innerlee commented Aug 5, 2021

innerlee commented Aug 5, 2021

codecov bot commented Jul 24, 2021 •

edited

Loading

innerlee Aug 5, 2021 •

edited

Loading