feat: add model script, training recipe and pretrained weight of halonet #720
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Songyuanwei
reviewed
Jul 31, 2023
configs/halonet/README.md
Outdated
| ## Introduction | ||
|
|
||
| Researchers from Google Research and UC Berkeley have developed a new model of self-attention that can outperform standard baseline models and even high-performance convolutional models.[[1](#references)] | ||
| Blocked Self-Attention：The whole input image is divided into multiple blocks and self-attention is applied to each block.However, if only the information inside the block is considered each time, it will inevitably lead to the loss of information.Therefore, before calculating the SA, a haloing operation is performed on each block, i.e., outside of each block, the information of the original image is used to padding a circle, so that the sensory field of each block can be appropriately larger and focus on more information. |
Songyuanwei
reviewed
Jul 31, 2023
configs/halonet/README.md
Outdated
|
|
||
| | Model | Context | Top-1 (%) | Top-5 (%) | Params (M) | Recipe | Download | | ||
| | ---------- | -------- | --------- | --------- | ---------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | ||
| | halonet50t | D910X8-G | 79.53 | 94.79 | 28.59 | [yaml](https://github.com/mindspore-lab/mindcv/blob/main/configs/halonet/halonet_50t_ascend.yaml) | [weights](https://download.mindspore.cn/toolkits/mindcv/halonet/halonet_50t-533da6be.ckpt) | |
Songyuanwei
reviewed
Jul 31, 2023
configs/halonet/README.md
Outdated
|
|
||
| | Model | Context | Top-1 (%) | Top-5 (%) | Params (M) | Recipe | Download | | ||
| | ---------- | -------- | --------- | --------- | ---------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | ||
| | halonet50t | D910X8-G | 79.53 | 94.79 | 28.59 | [yaml](https://github.com/mindspore-lab/mindcv/blob/main/configs/halonet/halonet_50t_ascend.yaml) | [weights](https://download.mindspore.cn/toolkits/mindcv/halonet/halonet_50t-533da6be.ckpt) | |
There was a problem hiding this comment.
模型参数是28.59吗？应该不是28.59. 看一下训练日志里面打印的模型参数信息
| # lr scheduler | ||
| scheduler: 'warmup_cosine_decay' | ||
| min_lr: 0.000006 | ||
| lr: 0.000234375 |
There was a problem hiding this comment.
lr好像不是这个吧。。0.00125？确认一下，其他参数也对照确认一下
mindcv/models/halonet.py
Outdated
|
|
||
|
|
||
| default_cfgs = { | ||
| "halonet50t": _cfg( |
There was a problem hiding this comment.
应该是halonet_50t吧。参考edgenext.py，_cfg里面加一下input_size=(3, 256, 256)
mindcv/models/halonet.py
Outdated
| def halonet_50t(pretrained: bool = False, num_classes: int = 1000, in_channels=3, **kwargs): | ||
| """Get HaloNet model. | ||
| Refer to the base class `models.HaloNet` for more details.""" | ||
| default_cfg = default_cfgs["halonet"] |
There was a problem hiding this comment.
和上面default_cfgs里面的名字对齐
mindcv/models/halonet.py
Outdated
| self.dim_out_qk, | ||
| 1, | ||
| stride=self.block_stride, | ||
|
|
mindcv/models/halonet.py
Outdated
| x = self.conv1(x) | ||
| x = self.conv2(x) | ||
| x = self.conv3(x) | ||
| x = ms.numpy.pad(x, ((0, 0), (0, 0), (1, 0), (1, 0)), constant_values=-32768) |
There was a problem hiding this comment.
这里就用你之前提交的代码。。init里面初始化self.pad， construct里面调用。。不要有这些奇奇怪怪的数字
Songyuanwei
reviewed
Aug 1, 2023
configs/halonet/README.md
Outdated
| @@ -0,0 +1,81 @@ | |||
| HaloNet | |||
Songyuanwei
changed the title
Songyuanwei
changed the title
mindcv/models/halonet.py
Outdated
| x = msnp.tensordot(q, rel_k, axes=1) | ||
| x = ops.reshape(x, (-1, W, rel_size)) | ||
| # pad to shift from relative to absolute indexing | ||
| pad = ops.Pad(paddings=((0, 0), (0, 0), (0, 1))) |
Songyuanwei
approved these changes
Aug 7, 2023
geniuspatrick
approved these changes
Aug 8, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thank you for your contribution to the MindCV repo.
Before submitting this PR, please make sure:
Motivation
(Write your motivation for proposed changes here.)
Test Plan
(How should this PR be tested? Do you require special setup to run the test or repro the fixed bug?)
Related Issues and PRs
(Is this PR part of a group of changes? Link the other relevant PRs and Issues here. Use https://help.github.com/en/articles/closing-issues-using-keywords for help on GitHub syntax)