Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding grounding dino #26087

Merged
merged 274 commits into from
Apr 11, 2024
Merged

Adding grounding dino #26087

Changes from 1 commit
Commits
Show all changes
274 commits
Select commit Hold shift + click to select a range
6e37211
Fixed typo when converting weigths to GroundingDINO vision backbone
EduardoPach Sep 18, 2023
0db05e0
Final modifications on modeling
EduardoPach Sep 20, 2023
a1eba2e
Removed unnecessary class
EduardoPach Sep 20, 2023
9cf7c3a
Fixed convert structure
EduardoPach Sep 20, 2023
9c55b24
Added image processing
EduardoPach Sep 24, 2023
ae570bb
make fixup partially completed
EduardoPach Sep 24, 2023
1f6475f
Now text_backbone_config has its own class
EduardoPach Oct 6, 2023
d763e04
Modified convert script
EduardoPach Oct 6, 2023
04022d4
Removed unnecessary config attribute
EduardoPach Oct 6, 2023
938f805
Added new function to generate sub sentence mask
EduardoPach Oct 13, 2023
6f08b04
Renamed parameters with gamma in the name as it's currently not allowed
EduardoPach Oct 13, 2023
7666253
Removed tokenization and image_processing scripts since we'll map fro…
EduardoPach Oct 13, 2023
046e0c5
Fixed some issues with configuration
EduardoPach Oct 13, 2023
70b248d
Just some modifications on conversion script
EduardoPach Oct 13, 2023
3bc92b7
Other modifications
EduardoPach Oct 13, 2023
4cae0ca
Copied deformable detr
EduardoPach Aug 22, 2023
149b462
First commit
EduardoPach Aug 23, 2023
92c31bf
Added bert to model
EduardoPach Aug 27, 2023
8f0a755
Bert validated
EduardoPach Aug 30, 2023
fb1c55c
Created Text and Fusion layers for Encoder
EduardoPach Aug 31, 2023
86131af
Adapted Encoder layer
EduardoPach Aug 31, 2023
8ad3226
Fixed typos
EduardoPach Sep 1, 2023
21e3fa2
Adjusted Encoder
EduardoPach Sep 4, 2023
5ddfa38
Converted encoder to hf
EduardoPach Sep 4, 2023
0512f7a
Modified Decoder Layer
EduardoPach Sep 5, 2023
d2cd35f
Modified main decoder class
EduardoPach Sep 6, 2023
cb2ad7f
Removed copy comments
EduardoPach Sep 6, 2023
eaf958d
Fixed forward from GroundingDINOModel and GroundingDINODecoder
EduardoPach Sep 11, 2023
88d07b3
Added all necessary layers, configurations and forward logic up to Gr…
EduardoPach Sep 12, 2023
f17bd3d
Added all layers to convertion
EduardoPach Sep 12, 2023
dcd1990
Fixed outputs for GroundingDINOModel and GroundingDINOForObjectDetection
EduardoPach Sep 12, 2023
39a161c
Fixed mask input to encoders and fixed nn.MultiheadAttention batch fi…
EduardoPach Sep 13, 2023
5ec72fb
Fixed forward from GroundingDINOTextEnhancerLayer
EduardoPach Sep 13, 2023
086f68a
Fixed output bug with GroundingDINODeformableLayer
EduardoPach Sep 13, 2023
f75cda2
Fixed bugs that prevent GroundingDINOForObjectDetection to run forwar…
EduardoPach Sep 15, 2023
8dbed3d
Fixed attentions to be passed correctly
EduardoPach Sep 18, 2023
a2af172
Passing temperature arg when creating Sine position embedding
EduardoPach Sep 18, 2023
759fc14
Removed copy comments
EduardoPach Sep 18, 2023
5196373
Added temperature argument for position embedding
EduardoPach Sep 18, 2023
900cff4
Fixed typo when converting weigths to GroundingDINO vision backbone
EduardoPach Sep 18, 2023
f23a54a
Final modifications on modeling
EduardoPach Sep 20, 2023
3090b2c
Removed unnecessary class
EduardoPach Sep 20, 2023
5c19e75
Fixed convert structure
EduardoPach Sep 20, 2023
aec2f68
Added image processing
EduardoPach Sep 24, 2023
b7a79cd
make fixup partially completed
EduardoPach Sep 24, 2023
685f1d6
Now text_backbone_config has its own class
EduardoPach Oct 6, 2023
d6e88fc
Modified convert script
EduardoPach Oct 6, 2023
0242e57
Removed unnecessary config attribute
EduardoPach Oct 6, 2023
af06c85
Added new function to generate sub sentence mask
EduardoPach Oct 13, 2023
43c0ce5
Renamed parameters with gamma in the name as it's currently not allowed
EduardoPach Oct 13, 2023
2bb7b70
Removed tokenization and image_processing scripts since we'll map fro…
EduardoPach Oct 13, 2023
98f3840
Fixed some issues with configuration
EduardoPach Oct 13, 2023
703eeff
Just some modifications on conversion script
EduardoPach Oct 13, 2023
c1c1467
Other modifications
EduardoPach Oct 13, 2023
bfb8829
Fix style
NielsRogge Oct 14, 2023
587589e
Improve fixup
NielsRogge Oct 14, 2023
f683611
Improve conversion script
NielsRogge Oct 14, 2023
3a0c742
Improve conversion script
NielsRogge Oct 14, 2023
6115547
Add GroundingDINOProcessor
NielsRogge Oct 14, 2023
cc1788f
More improvements
NielsRogge Oct 14, 2023
a6dea4a
Return token type ids
NielsRogge Oct 14, 2023
ae6e110
something
EduardoPach Oct 14, 2023
9fba8c2
Fix more tests
NielsRogge Oct 15, 2023
684a0bb
More improvements
NielsRogge Oct 15, 2023
3b2d576
More cleanup
NielsRogge Oct 15, 2023
88e5d02
More improvements
NielsRogge Oct 15, 2023
55390d1
Merge branch 'adding-grounding-dino' of https://github.com/EduardoPac…
EduardoPach Oct 16, 2023
8bae1bd
Fixed tests, improved modeling and config
EduardoPach Oct 16, 2023
f343f78
More improvements and fixing tests
EduardoPach Oct 17, 2023
033d903
Improved tests and modeling
EduardoPach Oct 18, 2023
baed29a
Improved tests and added image processor
EduardoPach Oct 21, 2023
50c5f67
Improved tests inference
EduardoPach Oct 22, 2023
d2922e1
More improvements
EduardoPach Oct 23, 2023
891c34d
More test improvements
EduardoPach Oct 26, 2023
eccaec9
Fixed last test
EduardoPach Oct 26, 2023
f32be01
Improved docstrings and comments
EduardoPach Oct 26, 2023
1c657e2
Fix style
NielsRogge Oct 27, 2023
1202ce8
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 1, 2023
d62dd11
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 1, 2023
bbf873b
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 1, 2023
c69b8a2
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 1, 2023
274752c
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 1, 2023
91373e0
Better naming
EduardoPach Nov 1, 2023
4945883
Better naming
EduardoPach Nov 1, 2023
5882f5f
Added Copied statement
EduardoPach Nov 1, 2023
c96a1a1
Added Copied statement
EduardoPach Nov 1, 2023
558ad87
Moved param init from GroundingDINOBiMultiHeadAttention
EduardoPach Nov 1, 2023
5c32bdc
Better naming
EduardoPach Nov 1, 2023
c561087
Fixing clamp style
EduardoPach Nov 1, 2023
07d4c62
Better naming
EduardoPach Nov 2, 2023
ba37183
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 2, 2023
c746e1d
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 2, 2023
07b260d
Update src/transformers/models/grounding_dino/configuration_grounding…
EduardoPach Nov 2, 2023
898e072
Update src/transformers/models/grounding_dino/convert_grounding_dino_…
EduardoPach Nov 2, 2023
34b36a3
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Nov 2, 2023
e14d6ae
Improving conversion script
EduardoPach Nov 2, 2023
f867e50
Improved config
EduardoPach Nov 2, 2023
fc105be
Improved naming
EduardoPach Nov 2, 2023
ed1176e
Improved naming again
EduardoPach Nov 2, 2023
ef5c90f
Improved grouding-dino.md
EduardoPach Nov 2, 2023
b2fd868
Moved grounding dino to multimodal
EduardoPach Nov 2, 2023
c23497c
Update src/transformers/models/grounding_dino/convert_grounding_dino_…
EduardoPach Nov 3, 2023
a729a38
Fixed docstrings and style
EduardoPach Nov 3, 2023
aafcc34
Fix docstrings
NielsRogge Nov 13, 2023
e4bad9b
Remove timm attributes
NielsRogge Nov 13, 2023
e48d411
Reorder imports
NielsRogge Nov 13, 2023
a7f026f
More improvements
NielsRogge Nov 13, 2023
1930b2a
Add Grounding DINO to pipeline
NielsRogge Nov 13, 2023
6ac265c
Remove model from check_repo
NielsRogge Nov 13, 2023
93b8609
Added grounded post_process to GroundingDINOProcessor
EduardoPach Nov 14, 2023
6461389
Fixed style
EduardoPach Nov 14, 2023
e35f1c9
Fixed GroundingDINOTextPrenetConfig docstrings
EduardoPach Nov 14, 2023
695ffa5
Aligned inputs.keys() when both image and text are passed with model_…
EduardoPach Nov 16, 2023
7d16d7f
Added tests for GroundingDINOImageProcessor and GroundingDINOProcessor
EduardoPach Nov 16, 2023
98321e3
Testing post_process_grounded_object_detection from GroundingDINOProc…
EduardoPach Nov 16, 2023
3da62df
Fixed order
EduardoPach Nov 18, 2023
6be9a68
Marked test with require_torch
EduardoPach Nov 18, 2023
cc1ee60
Temporarily changed repo_id
EduardoPach Nov 18, 2023
8cf167e
More improvements
EduardoPach Nov 18, 2023
27edb8e
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Nov 20, 2023
2927c13
Fix style
NielsRogge Nov 20, 2023
42ee6bc
Final improvements
EduardoPach Nov 20, 2023
85acfbc
Merge branch 'adding-grounding-dino' of https://github.com/EduardoPac…
EduardoPach Nov 20, 2023
e2b48b0
Improve annotators
NielsRogge Nov 23, 2023
5e1f0d9
Fix style
NielsRogge Nov 23, 2023
c9a8440
Add is_torch_available
NielsRogge Nov 23, 2023
f954f4b
Remove type hints
NielsRogge Nov 23, 2023
2eb2a98
vocab_tokens as one liner
EduardoPach Dec 8, 2023
625123a
Removed print statements
EduardoPach Dec 8, 2023
4553ad1
Renamed GroundingDINOTextPrenetConfig to GroundingDINOTextConfig
EduardoPach Dec 8, 2023
3b6b2c2
remove unnecessary comments
EduardoPach Dec 8, 2023
afb2649
Removed unnecessary tests on conversion script
EduardoPach Dec 8, 2023
4fdaf42
Renamed GroundingDINO to camel case GroundingDino
EduardoPach Dec 8, 2023
559de31
Fixed GroundingDinoProcessor docstrings
EduardoPach Dec 8, 2023
fef983e
loading MSDA kernels in the modeling file
EduardoPach Dec 8, 2023
fbf82be
Fix merge
NielsRogge Dec 11, 2023
9994ee0
Fix copies
NielsRogge Dec 11, 2023
14c839d
Replace nn.multiheadattention
NielsRogge Jan 31, 2024
5a6f258
Replace nn.multiheadattention
NielsRogge Feb 1, 2024
9fa83da
Fixed inputs for GroundingDinoMultiheadAttention & order of modules
Feb 4, 2024
06ba0ec
Fixed processing to avoid messing with inputs
Feb 4, 2024
9cda12e
Added more tips for GroundingDino
Feb 4, 2024
bde2c6a
Make style
Feb 4, 2024
01c382e
Chaning name to align with SAM
Feb 4, 2024
5d1f0e7
Replace final nn.multiheadattention
NielsRogge Feb 4, 2024
339915f
Fix model tests
NielsRogge Feb 4, 2024
1bb4886
Update year, remove GenerationTesterMixin
NielsRogge Feb 4, 2024
4bb58d3
Address comments
NielsRogge Feb 4, 2024
2c5d4ea
Address more comments
NielsRogge Feb 4, 2024
f21162c
Rename TextPrenet to TextModel
NielsRogge Feb 4, 2024
48f1734
Rename hidden_states
NielsRogge Feb 4, 2024
d3f45c3
Address more comments
NielsRogge Feb 4, 2024
3134d39
Address more comments
NielsRogge Feb 4, 2024
1485264
Address comment
NielsRogge Feb 4, 2024
c918fca
Merge branch 'adding-grounding-dino' of https://github.com/EduardoPac…
Feb 4, 2024
fc2251e
Merge branch 'adding-grounding-dino' of https://github.com/EduardoPac…
Feb 4, 2024
36c64be
Address more comments
NielsRogge Feb 4, 2024
8f338dd
Address merge
NielsRogge Feb 4, 2024
a46c4f0
Address comment
NielsRogge Feb 5, 2024
a8a6bea
Address comment
NielsRogge Feb 5, 2024
28686ec
Address comment
NielsRogge Feb 5, 2024
a3330ac
Make style
Feb 5, 2024
e9a45cb
Merge branch 'adding-grounding-dino' of https://github.com/EduardoPac…
Feb 5, 2024
21a1b4b
Added layer norm eps to layer norms
Feb 5, 2024
7292639
Address more comments
NielsRogge Feb 5, 2024
6e51931
More fixes
Feb 5, 2024
d5481bb
Fixed equivalence
Feb 5, 2024
1fcf142
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
NielsRogge Feb 5, 2024
098a59d
Make fixup
NielsRogge Feb 5, 2024
e005007
Remove print statements
NielsRogge Feb 5, 2024
daa29dc
Address comments
NielsRogge Feb 5, 2024
7d4c763
Address comments
NielsRogge Feb 7, 2024
f52dd2d
Address comments
NielsRogge Feb 7, 2024
afb5c6e
Address comments
NielsRogge Feb 8, 2024
4a88014
Address comments
NielsRogge Feb 8, 2024
34e37b4
Address comments
NielsRogge Feb 9, 2024
4854862
Add comment
NielsRogge Feb 9, 2024
c9fcadd
Address comment
NielsRogge Feb 9, 2024
580ce27
Fix merge
NielsRogge Feb 9, 2024
6366302
Remove overwriting of test
NielsRogge Feb 10, 2024
b5b1f1b
Fix bbox_embed
NielsRogge Feb 10, 2024
9faa6b4
Improve decoder_bbox_embed_share
NielsRogge Feb 10, 2024
e7761f7
Simplify outputs
NielsRogge Feb 10, 2024
09ae5c1
Updated post_process_grounded_object_detection
Feb 10, 2024
6036be6
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
Feb 12, 2024
9700e22
Renamed sources to feature_maps
Feb 13, 2024
66ebb6d
Improved tests for Grounding Dino ImageProcessor and Processor
Feb 15, 2024
1f4ffae
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
Feb 15, 2024
b27e7fb
Fixed test requirements and imports
Feb 15, 2024
17387df
Fixed image_processing
Feb 15, 2024
eed03aa
Fixed processor tests
Feb 15, 2024
3e8772a
Fixed imports for image processing tests
Feb 15, 2024
1682c0a
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
Feb 15, 2024
c549574
Fix copies
Feb 15, 2024
fdf7e82
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Feb 17, 2024
126dd83
Updated modeling
EduardoPach Feb 17, 2024
d24335b
Fix style
EduardoPach Feb 17, 2024
7d6bd5b
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Feb 21, 2024
0b8f4e8
Moved functions to correct position
EduardoPach Feb 21, 2024
eafd39f
Fixed copy issues
EduardoPach Feb 21, 2024
fb1e202
Update src/transformers/models/deformable_detr/modeling_deformable_de…
EduardoPach Feb 24, 2024
516de4a
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Feb 24, 2024
0ae3c5d
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Feb 24, 2024
7255f13
Keeping consistency custom cuda kernels for MSDA
EduardoPach Feb 24, 2024
2fb611a
Make GroundingDinoProcessor logic clearer
EduardoPach Feb 24, 2024
83004b7
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Feb 24, 2024
c7a4ef0
Updated Grounding DINO checkpoints
EduardoPach Feb 24, 2024
baa1959
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Feb 26, 2024
03137fd
Changed tests to correct structure
EduardoPach Mar 1, 2024
3ee2d78
Updated gpu-cpu equivalence test
EduardoPach Mar 4, 2024
8361ffc
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Mar 4, 2024
fcfad83
fix copies
EduardoPach Mar 4, 2024
ed7a71e
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Mar 5, 2024
fe7cd12
Update src/transformers/models/grounding_dino/processing_grounding_di…
EduardoPach Mar 9, 2024
ebf136f
Update src/transformers/models/grounding_dino/processing_grounding_di…
EduardoPach Mar 9, 2024
8728db6
Update src/transformers/models/grounding_dino/modeling_grounding_dino.py
EduardoPach Mar 9, 2024
1be93d6
Update src/transformers/models/grounding_dino/configuration_grounding…
EduardoPach Mar 9, 2024
0c99ac7
Fixed erros and style
EduardoPach Mar 10, 2024
538e88f
Fix copies
EduardoPach Mar 10, 2024
18d3d63
Removed inheritance from PreTrainedModel from GroundingDinoTextModel
EduardoPach Mar 10, 2024
b4735d5
Fixed GroundingDinoTextModel
EduardoPach Mar 10, 2024
1cf5cf0
Fixed type of default backbone config
EduardoPach Mar 10, 2024
88c0467
Fixed missing methods for GroundingDinoTextModel and Added timm suppo…
EduardoPach Mar 10, 2024
2d95044
Addressed comments
EduardoPach Mar 10, 2024
710c1be
Addressed batched image processing tests
EduardoPach Mar 10, 2024
06a59b2
Addressed zero shot test comment
EduardoPach Mar 10, 2024
2de4e15
Addressed tip comment
EduardoPach Mar 10, 2024
0780569
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Mar 11, 2024
4b9c9ad
Removed GroundingDinoTextModel from check_repo
EduardoPach Mar 22, 2024
4df56a4
Removed inplace masking
EduardoPach Mar 22, 2024
0e0ae3c
Addressed comments
EduardoPach Mar 22, 2024
e8222f3
Addressed comments
EduardoPach Mar 22, 2024
6cab49a
Addressed comments
EduardoPach Mar 22, 2024
8012f13
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Mar 22, 2024
37b272f
Fix copies
EduardoPach Mar 22, 2024
d6966ce
Fixing timm test
EduardoPach Mar 23, 2024
1a94461
Fixed batching equivalence test
EduardoPach Mar 23, 2024
a584f65
Update docs/source/en/model_doc/grounding-dino.md
EduardoPach Mar 24, 2024
a9dfee3
Update docs/source/en/model_doc/grounding-dino.md
EduardoPach Mar 24, 2024
6f13fbb
Update docs/source/en/model_doc/grounding-dino.md
EduardoPach Mar 24, 2024
2f845b0
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Apr 4, 2024
a1e9ff0
Addressed more comments
EduardoPach Apr 8, 2024
38a2e97
Added a new comment
EduardoPach Apr 9, 2024
e9633b4
Reduced image size
EduardoPach Apr 9, 2024
89e070f
Addressed more comments
EduardoPach Apr 9, 2024
a961ab7
Nits
EduardoPach Apr 10, 2024
6c2a617
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
EduardoPach Apr 10, 2024
f945c7a
Nits
EduardoPach Apr 10, 2024
b0891ca
Changed the way text_config is initialized
EduardoPach Apr 10, 2024
c630a9c
Update src/transformers/models/grounding_dino/processing_grounding_di…
EduardoPach Apr 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge remote-tracking branch 'upstream/main' into adding-grounding-dino
  • Loading branch information
Eduardo Pacheco committed Feb 15, 2024
commit 1f4ffae28cfb00626bf56e655ff1ffdb3a788a5c

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.