[Feature] add DPT head #605

xiexinch · 2021-06-17T09:03:55Z

Implementation of Vision Transformers for Dense Prediction.
Modified from the official repository.

Training results:

backbone	mIoU	mIoU (ms+flip)
ViT-B	46.97	48.34
ViT-L	46.19	46.97

Different from the original paper, I train with AdamW optimizer instead of SGD.

codecov · 2021-06-17T09:18:13Z

Codecov Report

Merging #605 (522cdff) into master (5753f41) will increase coverage by 0.13%.
The diff coverage is 96.19%.

@@            Coverage Diff             @@
##           master     #605      +/-   ##
==========================================
+ Coverage   88.76%   88.90%   +0.13%     
==========================================
  Files         109      110       +1     
  Lines        5823     5928     +105     
  Branches      935      950      +15     
==========================================
+ Hits         5169     5270     +101     
- Misses        463      465       +2     
- Partials      191      193       +2

Flag	Coverage Δ
unittests	`88.88% <96.19%> (+0.13%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmseg/models/decode_heads/dpt_head.py	`96.15% <96.15%> (ø)`
mmseg/models/decode_heads/__init__.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5753f41...522cdff. Read the comment docs.

mmseg/models/decode_heads/dpt.py

mmseg/models/utils/post_process_layer.py

… dpt

mmseg/models/backbones/vit.py

configs/dpt/dpt_vit-l16_512x512_160k_ade20k.py

mmseg/models/decode_heads/dpt_head.py

configs/dpt/README.md

mmseg/models/decode_heads/dpt_head.py

Junjun2016

Add this method to Supported methods (Chinese and English) https://github.com/open-mmlab/mmsegmentation#benchmark-and-model-zoo.

README.md

Junjun2016

Please fix the conflict.

mmseg/models/decode_heads/dpt_head.py

xvjiarui · 2021-08-28T18:53:48Z

LGTM except for one comment.

Please resolve the conflict btw.

Junjun2016 · 2021-08-30T05:09:29Z

Check checkpoints' keys, maybe you should update keys.

* add DPT head * [fix] fix init error * use mmcv function * delete code * remove transpose clas * support NLC output shape * Delete post_process_layer.py * add unittest and docstring * rename variables * fix project error and add unittest * match dpt weights * add configs * fix vit pos_embed bug and dpt feature fusion bug * match vit output * fix gelu * minor change * update unitest * fix configs error * inference test * remove auxilary * use local pretrain * update training results * update yml * update fps and memory test * update doc * update readme * add yml * update doc * remove with_cp * update config * update docstring * remove dpt-l * add init_cfg and modify readme.md * Update dpt_vit-b16.py * zh-n README * use constructor instead of build function * prevent tensor being modified by ConvModule * fix unittest Co-authored-by: Junjun2016 <hejunjun@sjtu.edu.cn>

* Add configs and some stats. * Update. * Update r101 8x8. * Minor. * Update url. * Update changelog. * Fix typos and workdir.

add DPT head

46dce78

xiexinch requested review from Junjun2016, xvjiarui and clownrat6 June 17, 2021 09:03

[fix] fix init error

7b80fd0