Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support Side Adapter Network #3232

Merged
merged 67 commits into from
Sep 20, 2023
Merged

[Feature] Support Side Adapter Network #3232

Changes from 1 commit
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
48344d2
add vit configs
angiecao Jul 25, 2023
236c1fe
Add AdapterSideNetwork decodehead
angiecao Jul 25, 2023
41b8db9
Add text_encoder in CLIP
angiecao Jul 25, 2023
0213604
Add configs for SAN
angiecao Jul 25, 2023
0ec5238
Add model covert for SAN
angiecao Jul 25, 2023
5b3b9e6
fix circular import
angiecao Jul 25, 2023
7de7c0c
fix bg_embed
angiecao Jul 25, 2023
3fac1ec
add test units for SAN
angiecao Jul 25, 2023
16662d8
Add regex, ftfy requirements
angiecao Jul 27, 2023
340688f
change 'multimodel' to 'multimodal'
angiecao Jul 27, 2023
1fffd59
Replace file_client with fileio
angiecao Jul 27, 2023
c992216
add vit configs
angiecao Jul 25, 2023
75811e3
Add AdapterSideNetwork decodehead
angiecao Jul 25, 2023
91dcfa1
Add text_encoder in CLIP
angiecao Jul 25, 2023
c7f3167
Add configs for SAN
angiecao Jul 25, 2023
6c2f01e
Add model covert for SAN
angiecao Jul 25, 2023
c0cad61
fix circular import
angiecao Jul 25, 2023
3e7bcca
fix bg_embed
angiecao Jul 25, 2023
7fb34b6
add test units for SAN
angiecao Jul 25, 2023
125bf17
Add regex, ftfy requirements
angiecao Jul 27, 2023
7b82303
change 'multimodel' to 'multimodal'
angiecao Jul 27, 2023
8f01fb4
freeze parameters
angiecao Jul 27, 2023
2ed4f6c
weight init
angiecao Jul 27, 2023
34a230c
add convert CLIP model into mmseg style
angiecao Jul 27, 2023
7229c00
train_pipeline & optimizer & scheduler
angiecao Jul 27, 2023
53573a6
add deep supervised
angiecao Jul 29, 2023
3e3bbea
add multimodal dependencies
angiecao Aug 1, 2023
6449203
Merge branch 'angiecao/add_SAN_infer' of github.com:angiecao/mmsegmen…
angiecao Aug 1, 2023
a327d70
fix test_san_head
angiecao Aug 1, 2023
887ae9f
[CodeCamp2023-154] Add semantic label to the segmentation visualizati…
CastleDream Aug 1, 2023
6afe346
add train loss for SAN
angiecao Aug 1, 2023
da69d7e
fix weights_init
angiecao Aug 2, 2023
790add5
[Fix] Fix module PascalContextDataset (#3235)
angiecao Aug 9, 2023
8997794
[Fix] Added ignore_index and one hot encoding for dice loss (#3237)
yeedrag Aug 9, 2023
759bd7e
[CodeCamp2023-367] Add pp_mobileseg model (#3239)
Yang-Changhui Aug 9, 2023
0c5db50
[Project] Support CAT-Seg from CVPR2023 (#3098)
SheffieldCao Aug 9, 2023
aabc3ee
[Fix] update pp_mobileseg ckpt links (#3254)
xiexinch Aug 10, 2023
f8ecb4b
[Doc]fix inference_segmentor to inference_model (#3261)
ooooo-create Aug 15, 2023
7bec537
fix dataset configs
angiecao Aug 16, 2023
03071cb
add assert & remove unused code
angiecao Aug 16, 2023
5b1ae9c
fix conflits with dev-1.x
angiecao Aug 16, 2023
327e757
fix conflits with dev-1.x
angiecao Aug 16, 2023
487827a
merge infer to train
angiecao Aug 31, 2023
48ccb01
fix loss
angiecao Aug 31, 2023
dd0b717
fix conflicts with add_SAN_train
angiecao Aug 31, 2023
54f8912
delete unused files
angiecao Aug 31, 2023
a459ddf
fix no valid category of ground truth
angiecao Sep 1, 2023
9a9a505
change to naive dice loss & set weight decay to 1e-4
angiecao Sep 2, 2023
fdf6617
add amp&clip grad
angiecao Sep 5, 2023
b0e9d22
fix loss_mask_ce is nan when use amp
angiecao Sep 5, 2023
89600c9
fix loss_cls_ce
angiecao Sep 6, 2023
16c65fc
fix error when use ViT/L-14 in clip2mmseg
angiecao Sep 7, 2023
a09c41a
Revert "fix loss_cls_ce"
angiecao Sep 7, 2023
4954fc8
fix avg_factor when class_weight is not None
angiecao Sep 7, 2023
ea2abc9
change num_total_masks to float32
angiecao Sep 7, 2023
9362e12
add assertion for InstanceData in match_cost
angiecao Sep 7, 2023
d69dc51
add test units for train SAN
angiecao Sep 7, 2023
f3e69a6
change batchsize for train
angiecao Sep 7, 2023
28b741f
remove unused & add reference link & change LayerNorm to LayerNorm2d
angiecao Sep 13, 2023
8a18f37
Merge branch 'dev-1.x' into angiecao/add_SAN_infer
angiecao Sep 15, 2023
e45845d
fix pre-commit error
angiecao Sep 15, 2023
fc5424e
combine two cross attention methods into one
angiecao Sep 20, 2023
da670c7
fix docstring
angiecao Sep 20, 2023
d30d37f
Merge branch 'angiecao/add_SAN_infer' of github.com:angiecao/mmsegmen…
angiecao Sep 20, 2023
8e50943
modify pretrained_Part load method & replace reduce_mean with all_reduce
angiecao Sep 20, 2023
17a4279
add readme.md
angiecao Sep 20, 2023
75a6338
add readme.md
angiecao Sep 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
change num_total_masks to float32
  • Loading branch information
@angiecao
angiecao committed Sep 7, 2023
commit ea2abc90bcb0de24bf52f4d2f5573db01b6dcb9f
5 changes: 3 additions & 2 deletions mmseg/models/decode_heads/san_head.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -660,7 +660,8 @@ def loss_by_feat(
cls_scores = cls_scores.flatten(0, 1)
labels = labels.flatten(0, 1)

num_total_masks = reduce_mean(cls_scores.new_tensor([avg_factor]))
num_total_masks = reduce_mean(
cls_scores.new_tensor([avg_factor], dtype=torch.float))
num_total_masks = max(num_total_masks, 1)

# extract positive ones
Expand Down Expand Up @@ -704,7 +705,7 @@ def loss_by_feat(
loss[loss_decode.loss_name] = loss_decode(
mask_point_preds,
mask_point_targets,
avg_factor=num_total_masks.float() *
avg_factor=num_total_masks *
self.train_cfg.num_points)
elif loss_decode.loss_name == 'loss_mask_dice':
loss[loss_decode.loss_name] = loss_decode(
Expand Down