Detection Transformer Estimator

[kieranfraser]

Description

Adding DEtection TRansformer (DETR) to ART to facilitate object detection using Transformers.

• DETR estimator added to art/estimators/object_detection
• requires additional method in art/estimators/pytorch.py which sets MultiheadAttention layer into train/eval mode
• associated tests added
• notebook demonstrating use of DETR estimator detecting objects and under adversarial patch attack added to notebooks/adversarial_patch/ and updated notebooks/ README

Paper: "End-to-End Object Detection with Transformers"

Fixes # (issue)
#1638

Type of change

Please check all relevant options.

• Improvement (non-breaking)
• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

• Test prediction of DETR
• Test loss gradient
• Test applying preprocessing defences
• Test computation of losses (classification, bbox, giou)
• Test DETR with PGD attack

Test Configuration:

• OS
• Python version
• ART version or commit number
• TensorFlow / Keras / PyTorch / MXNet version

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[codecov-commenter]

Codecov Report

Merging #2192 (81408e5) into dev_1.15.0 (28a3f8c) will decrease coverage by 0.05%.
The diff coverage is 69.19%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@              Coverage Diff               @@
##           dev_1.15.0    #2192      +/-   ##
==============================================
- Coverage       85.66%   85.61%   -0.05%     
==============================================
  Files             306      308       +2     
  Lines           27022    27448     +426     
  Branches         4962     5044      +82     
==============================================
+ Hits            23148    23500     +352     
- Misses           2602     2668      +66     
- Partials         1272     1280       +8     

Impacted Files	Coverage Δ
.../object_detection/pytorch_detection_transformer.py	63.26% <63.26%> (ø)
art/estimators/object_detection/detr.py	73.87% <73.87%> (ø)
art/estimators/object_detection/__init__.py	100.00% <100.00%> (ø)
art/estimators/pytorch.py	85.71% <100.00%> (+2.09%)	⬆️

... and 8 files with indirect coverage changes

[beat-buesser]

Hi @kieranfraser Thank you very much for your PR. I think we need to include the license of reused code somewhere. I think the patches don't have a lot of effects on the detections, is there a way how we could generate a stronger patch?

[kieranfraser]

Hi @kieranfraser Thank you very much for your PR. I think we need to include the license of reused code somewhere. I think the patches don't have a lot of effects on the detections, is there a way how we could generate a stronger patch?

Thanks @beat-buesser. I have added references to the license (Apache 2.0) in the Docstrings of reused code. Also updated the notebook with stronger Adversarial Patch attacks (targeted and untargeted). Seems that unless a fixed patch location is passed, the attack fails to converge to a patch that fools the DETR - plan to experiment further with this. But the notebook now demonstrates working patches as well as PGD for the DETR.