Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] timm backbones wrapper #427

Merged
merged 3 commits into from
Sep 6, 2021
Merged

[Feature] timm backbones wrapper #427

merged 3 commits into from
Sep 6, 2021

Conversation

amirassov
Copy link
Contributor

Motivation

TIMM (https://github.com/rwightman/pytorch-image-models) contains a lot of backbones, which are not available in this repository. But mmclassification code is more flexible for training and adding new components. It would be convenient to have wrapper here to use TIMM backbones.

Modification

This PR adds TIMMBackbone wrapper to use timm backbones.

Use cases (Optional)

To use Efficientnetv2-s from timm with pretrained ImageNet-21k weights you need to use the following backbone config:

backbone=dict(
    type="TIMMBackbone",
    pretrained=True,
    model_name="tf_efficientnetv2_s_in21k",
    init_cfg=None
)

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
  • CLA has been signed and all committers have signed the CLA in this PR.

@CLAassistant
Copy link

CLAassistant commented Aug 31, 2021

CLA assistant check
All committers have signed the CLA.

@amirassov amirassov changed the title Feature/timm backbone [Feature] timm backbones wrapper Aug 31, 2021
@mzr1996
Copy link
Member

mzr1996 commented Sep 1, 2021

Thanks for your contribution!
We will review this PR recently.
Usually, our backbones need to support downstream repos, so we can't use timm backbone directly. But if someone just want to use our platform to train third-party backbones, this function will be helpful.

@mzr1996
Copy link
Member

mzr1996 commented Sep 2, 2021

Hello, here is a problem that breaks the Python3.6 CI.

Python 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import timm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/xxx/miniconda3/envs/py36/lib/python3.6/site-packages/timm/__init__.py", line 2, in <module>
    from .models import create_model, list_models, is_model, list_modules, model_entrypoint, \
  File "/xxx/miniconda3/envs/py36/lib/python3.6/site-packages/timm/models/__init__.py", line 1, in <module>
    from .byoanet import *
  File "/xxx/miniconda3/envs/py36/lib/python3.6/site-packages/timm/models/byoanet.py", line 16, in <module>
    from .byobnet import ByoBlockCfg, ByoModelCfg, ByobNet, interleave_blocks
  File "/xxx/miniconda3/envs/py36/lib/python3.6/site-packages/timm/models/byobnet.py", line 28, in <module>
    from dataclasses import dataclass, field, replace
ModuleNotFoundError: No module named 'dataclasses'

timm uses dataclasses which is added in Python 3.7.

@codecov
Copy link

codecov bot commented Sep 6, 2021

Codecov Report

Merging #427 (59cd4ff) into master (729c6c1) will increase coverage by 0.01%.
The diff coverage is 81.81%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #427      +/-   ##
==========================================
+ Coverage   77.02%   77.04%   +0.01%     
==========================================
  Files          99      100       +1     
  Lines        5280     5302      +22     
  Branches      866      869       +3     
==========================================
+ Hits         4067     4085      +18     
- Misses       1093     1096       +3     
- Partials      120      121       +1     
Flag Coverage Δ
unittests 77.04% <81.81%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmcls/models/backbones/timm_backbone.py 80.95% <80.95%> (ø)
mmcls/models/backbones/__init__.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 729c6c1...59cd4ff. Read the comment docs.

Copy link
Member

@mzr1996 mzr1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mzr1996 mzr1996 self-requested a review September 6, 2021 03:05
@mzr1996 mzr1996 merged commit 5cfaed6 into open-mmlab:master Sep 6, 2021
@amirassov amirassov deleted the feature/timm-backbone branch September 6, 2021 07:35
shinya7y added a commit to shinya7y/mmdetection that referenced this pull request Jan 16, 2022
ZwwWayne pushed a commit to open-mmlab/mmdetection that referenced this pull request Feb 11, 2022
* add TIMMBackbone

based on
open-mmlab/mmpretrain#427
open-mmlab/mmsegmentation#998

* update and clean

* fix unit test

* Revert

* add example configs
chhluo pushed a commit to chhluo/mmdetection that referenced this pull request Feb 21, 2022
* add TIMMBackbone

based on
open-mmlab/mmpretrain#427
open-mmlab/mmsegmentation#998

* update and clean

* fix unit test

* Revert

* add example configs
ZwwWayne added a commit to open-mmlab/mmdetection that referenced this pull request Feb 26, 2022
* [Enhancement] Upgrade isort in pre-commit hook (#7130)

* upgrade isort to v5.10.1

* replace known_standard_library with extra_standard_library

* upgrade isort to v5.10.1

replace known_standard_library with extra_standard_library

* imports order changes

* [Fix] cannot to save the best checkpoint when the key_score is None (#7101)

* [Fix] Fix MixUp transform filter boxes failing case. Added test case (#7080)

* [Fix] Update the version limitation of mmcv-full and pytorch in CI. (#7133)

* Update

* Update build.yml

* Update build.yml

* [Feature] Support TIMMBackbone (#7020)

* add TIMMBackbone

based on
open-mmlab/mmpretrain#427
open-mmlab/mmsegmentation#998

* update and clean

* fix unit test

* Revert

* add example configs

* Create 2_new_data_model.md (#6476)

fix some typo

Co-authored-by: PJLAB\huanghaian <1286304229@qq.com>

* [FIX] add Ci of pytorch 1.10 and comments for bbox clamp (#7081) (#7083)

* add comments for bbox clamp

* add CI of pytorch1.10

* add ci of pytorch1.10.1

* mmcv1.9.0->mmcv1.9

* add ci of pytorch1.10

* Add daily issue owners (#7163)

* Add code owners

Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com>

* Update code owners

Signed-off-by: del-zhenwu <dele.zhenwu@gmail.com>

* [Feature] Support visualization for Panoptic Segmentation (#7041)

* First commit of v2

* split the functions

* Support to show panoptic result

* temp

* Support to show gt

* support show gt

* fix lint

* Support to browse datasets

* Fix unit tests

* Fix findContours

* fix comments

* Fix pre-commit

* fix lint

* Add the type of an argument

* [Fix] confusion_matrix.py analysis tool handling NaNs (#7147)

* [Fix] Added missing property in SABLHead (#7091)

* Added missing property in SABLHead

* set pre-commit-hooks to v0.1.0

* set maskdownlint to v0.11.0

* pre-commit-hooks

Co-authored-by: Cedric Luo <luochunhua1996@outlook.com>

* Update config.md (#7215)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Update config.md

fix some typos

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>

* [Feature] Support DyHead (#6823)

* add DyHead

* move and update DYReLU

* update

* replace stack with sum to reduce memory

* clean and update

* update to align inference accuracy (incomplete)

* fix pad

* update to align training accuracy and pick #6867

* add README and metafile

* update docs

* resolve comments

* revert picking 6867

* update README.md

* update metafile.yml

* resolve comments and update urls

* Fix broken colab link (#7218)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Fix broken colab link

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>

* Remove the inplace addition in `FPN` (#7175)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Remove the inplace addition in `FPN`

* update

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: PJLAB\huanghaian <1286304229@qq.com>

* [Feature] Support OpenImages Dataset (#6331)

* [Feature] support openimage group of eval

* [Feature] support openimage group of eval

* support openimage dataset

* support openimage challenge dataset

* fully support OpenImages-V6 and OpenImages Challenge 2019

* Fix some logic error

* update config file

* fix get data_infos error

* fully support OpenImages evaluation

* update OpenImages config files

* [Feature] support OpenImages datasets

* fix bug

* support load image metas from pipeline

* fix bug

* fix get classes logic error

* update code

* support get image metas

* support openimags

* support collect image metas

* support Open Images

* fix openimages logic

* minor fix

* add a new function to compute openimages tpfp

* minor fix

* fix ci error

* minor fix

* fix indication

* minor fix

* fix returns

* fix returns

* fix returns

* fix returns

* fix returns

* minor fix

* update readme

* support loading image level labels and fix some logic

* minor fix

* minor fix

* add class names

* minor fix

* minor fix

* minor fix

* add openimages test unit

* minor fix

* minor fix

* fix test unit

* minor fix

* fix logic error

* minor fix

* fully support openimages

* minor fix

* fix docstring

* fix docstrings in readthedocs

* update get image metas script

* label_description_file -> label_file

* update openimages readme

* fix test unit

* fix test unit

* minor fix

* update readme file

* Update get_image_metas.py

* [Enhance] Speed up SimOTA matching. (#7098)

* [Feature] Add Maskformer to mmdet (#7212)

* first commit

* add README

* move model description from config to readme

add description for binary_input

add description for dice loss

add a independent panoptic gt processing function

add a independent panoptic gt processing function

remove compatibility of pretrain in maskformer

* update comments in maskformer_head

* update docs format

* Add deprecation message for deploy tools (#7242)

* Add CI for windows (#7228)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Add mmrotate (#7216)

* fix description for args in CSPDarknet (#7187)

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* Update build.yml

* fix test_find_latest_checkpoint

* fix data_infos__default_db_directories

* fix test_custom_classes_override_default

* Update test_custom_dataset.py

* Update test_common.py

* Update assign_result.py

* use os.path.join

* fix bug

* Update test_common.py

* Update assign_result.py

* Update sampling_result.py

* os.path -> osp

* os.path -> osp

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: Hyeokjoon Kwon <dsazz801@gmail.com>

* add Chinese version of init_cfg (#7188)

* [Fix] Fix wrong img name in onnx2tensorrt.py (#7157)

* [Docs] fix albumentations installed way (#7143)

* Create init_cfg.md

* Update docs/zh_cn/tutorials/init_cfg.md

Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com>

* update init_cfg.md

* update init_cfg.md

* update init_cfg.md

* update init_cfg.md

Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: Haian Huang(深度眸) <1286304229@qq.com>

* update MaskFormer readme and docs (#7241)

* update docs for maskformer

* update readme

* update readme format

* update link

* update json link

* update format of ConfigDict

* update format of function returns

* uncomment main in deployment/test.py

* [Feature] ResNet Strikes Back. (#7001)

* [Feature] ResNet Strikes Back.

* add more cfg

* add readme

* update

* update

* update

* update

* update

* update

* Maskformer Visualization (#7247)

* maskformer visualization

* compatible with models that do not contain arg of pretrained

* compatible with models that do not contain arg of pretrained

* Bump versions to v2.22.0 (#7240)

* Bump versions to v2.22.0

* Fix comments and add the latest PRs

* fix the id of contributor

* relax the version of mmcv

* Add ResNet Strikes Back

* Update README_zh-CN.md

* Update README.md

* fix typo

* Update README_zh-CN.md

Co-authored-by: Wenwei Zhang <40779233+ZwwWayne@users.noreply.github.com>

* Maskformer metafile and rsb readme format (#7250)

* [Fix] Fix Open Images testunit to avoid error in Windows CI (#7252)

* [Feature] support openimage group of eval

* [Feature] support openimage group of eval

* support openimage dataset

* support openimage challenge dataset

* fully support OpenImages-V6 and OpenImages Challenge 2019

* Fix some logic error

* update config file

* fix get data_infos error

* fully support OpenImages evaluation

* update OpenImages config files

* [Feature] support OpenImages datasets

* fix bug

* support load image metas from pipeline

* fix bug

* fix get classes logic error

* update code

* support get image metas

* support openimags

* support collect image metas

* support Open Images

* fix openimages logic

* minor fix

* add a new function to compute openimages tpfp

* minor fix

* fix ci error

* minor fix

* fix indication

* minor fix

* fix returns

* fix returns

* fix returns

* fix returns

* fix returns

* minor fix

* update readme

* support loading image level labels and fix some logic

* minor fix

* minor fix

* add class names

* minor fix

* minor fix

* minor fix

* add openimages test unit

* minor fix

* minor fix

* fix test unit

* minor fix

* fix logic error

* minor fix

* fully support openimages

* minor fix

* fix docstring

* fix docstrings in readthedocs

* update get image metas script

* label_description_file -> label_file

* update openimages readme

* fix test unit

* fix test unit

* minor fix

* update readme file

* Update get_image_metas.py

* fix oid testunit to avoid some error in windows

* minor fix to avoid some error in windows

* minor fix

* add comments in oid test unit

* minor fix

Co-authored-by: Cedric Luo <luochunhua1996@outlook.com>
Co-authored-by: LuooChen <33743370+LuooChen@users.noreply.github.com>
Co-authored-by: Daniel van Sabben Alsina <13849741+dvansa@users.noreply.github.com>
Co-authored-by: jbwang1997 <jbwang1997@gmail.com>
Co-authored-by: Yosuke Shinya <42844407+shinya7y@users.noreply.github.com>
Co-authored-by: siatwangmin <csustwangmin@163.com>
Co-authored-by: PJLAB\huanghaian <1286304229@qq.com>
Co-authored-by: del-zhenwu <dele.zhenwu@gmail.com>
Co-authored-by: Guangchen Lin <347630870@qq.com>
Co-authored-by: VIKASH RANJAN <ranjanvikash26@gmail.com>
Co-authored-by: Range King <RangeKingHZ@gmail.com>
Co-authored-by: Jamie <jamiechoi1995@users.noreply.github.com>
Co-authored-by: BigDong <yudongwang@tju.edu.cn>
Co-authored-by: Haofan Wang <frankmiracle@outlook.com>
Co-authored-by: Zhijian Liu <zhijianliu.cs@gmail.com>
Co-authored-by: BigDong <yudongwang1226@gmail.com>
Co-authored-by: RangiLyu <lyuchqi@gmail.com>
Co-authored-by: Yue Zhou <592267829@qq.com>
Co-authored-by: Hyeokjoon Kwon <dsazz801@gmail.com>
Co-authored-by: Kevin Ye <1752391457@qq.com>
Chun-Yi-Wu pushed a commit to kneron/kneron-mmdetection that referenced this pull request Jul 18, 2022
* add TIMMBackbone

based on
open-mmlab/mmpretrain#427
open-mmlab/mmsegmentation#998

* update and clean

* fix unit test

* Revert

* add example configs
ZwwWayne pushed a commit to open-mmlab/mmdetection that referenced this pull request Jul 18, 2022
* add TIMMBackbone

based on
open-mmlab/mmpretrain#427
open-mmlab/mmsegmentation#998

* update and clean

* fix unit test

* Revert

* add example configs
ZwwWayne pushed a commit to ZwwWayne/mmdetection that referenced this pull request Jul 19, 2022
* add TIMMBackbone

based on
open-mmlab/mmpretrain#427
open-mmlab/mmsegmentation#998

* update and clean

* fix unit test

* Revert

* add example configs
mzr1996 added a commit to mzr1996/mmpretrain that referenced this pull request Nov 24, 2022
* Add wrapper to use backbones from timm

* Add tests

* Remove timm from optional deps and modify GitHub workflow.

Co-authored-by: mzr1996 <mzr1996@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants