A Repository of the Papers Addressing Imbalance Problems in Object Detection

This repository provides an up-to-date the list of studies addressing imbalance problems in object detection. It follows the taxonomy provided in the following paper (please cite the paper if you benefit from this repository):

K. Oksuz, B. C. Cam, S. Kalkan, E. Akbas, "Imbalance Problems in Object Detection: A Review", Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020.[preprint]

BibTeX entry:

@ARTICLE{imbalance,
       author = {Kemal Oksuz and Baris Can Cam and Sinan Kalkan and Emre Akbas},
        title = "{Imbalance Problems in Object Detection: A Review}",
      journal = {Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
         year = "2020",
        pages = {1-1}
        }

How to request addition of a paper

If you know of a paper that addresses an imbalance problem concerning generic object detection and is not on this repository, you are welcome to request the addition of that paper by submitting a pull request. In your pull request please briefly state which section of your paper is related to which problem.

Following the methodology in our paper, the papers should be designed for the generic object detection problem (i.e. reporting results on generic object detection datasets such as ILSVRC, Pascal VOC, MS-COCO, Open Images, Objects 365 etc.).

Table of Contents (Follows the taxonomy in the paper)

1. Class Imbalance
   1.1 Foreground-Background Class Imbalance
   1.2 Foreground-Foreground Class Imbalance
2. Scale Imbalance
   2.1 Object/box-level Scale Imbalance
   2.2 Feature-level Imbalance
3. Spatial Imbalance
   3.1 Imbalance in Regression Loss
   3.2 IoU Distribution Imbalance
   3.3 Object Location Imbalance
4. Objective Imbalance

1. Class Imbalance

1.1. Foreground-Background Class Imbalance

• Hard Sampling Methods

  • Random Sampling
  • Hard Example Mining
    • Bootstrapping, NeurIPS 1996, [paper]
    • SSD, ECCV 2016, [paper]
    • Online Hard Example Mining, CVPR 2016, [paper]
    • IoU-based Sampling, CVPR 2019, [paper]
    • Overlap Sampler, WACV 2020, [paper]
  • Limit Search Space
    • Two-stage Object Detectors
    • IoU-lower Bound, ICCV 2015, [paper]
    • Objectness Prior, CVPR 2017, [paper]
    • Negative Anchor Filtering, CVPR 2018, [paper]
    • Objectness Module, ICCV 2019, [paper]

• Soft Sampling Methods

  • Focal Loss, ICCV 2017, [paper]
  • Gradient Harmonizing Mechanism, AAAI 2019, [paper]
  • Prime Sample Attention, CVPR 2020, [paper]
  • Unified Sample Weighting Network, CVPR 2020, [paper]
  • Equalization Loss, CVPR 2020, [paper]
  • Quality Focal Loss, NeurIPS 2020, [paper]
  • Equalization Loss v2, CVPR 2021, [paper]

• Sampling-Free Methods

  • Is Sampling Heuristics Necessary in Training Deep Object Detectors?, arXiv 2019, [paper]
  • Residual Objectness for Imbalance Reduction, arXiv 2019, [paper]
  • AP Loss, CVPR 2019, [paper]
  • DR Loss, CVPR 2020, [paper]
  • aLRP Loss, NeurIPS 2020, [paper]
  • RS Loss, ICCV 2021, [paper]

• Generative Methods

  • Adversarial Faster-RCNN, CVPR 2017, [paper]
  • Task Aware Data Synthesis, CVPR 2019, [paper]
  • PSIS, arXiv 2019, [paper]
  • pRoI Generator, WACV 2020, [paper]

1.2. Foreground-Foreground Class Imbalance

• Fine-tuning Long Tail Distribution for Obj.Det., CVPR 2016, [paper]
• PSIS, arXiv 2019, [paper]
• OFB Sampling, WACV 2020, [paper]
• Large-Scale Object Detection in the Wild from Imbalanced Multi-Labels, CVPR 2020, [paper]
• Balanced Group Softmax Loss, CVPR 2020, [paper]
• SimCal, ECCV 2020, [paper]

2. Scale Imbalance

2.1. Object/box-level Scale Imbalance

• Methods Predicting from the Feature Hierarchy of Backbone Features

  • Scale-dependent Pooling, CVPR 2016, [paper]
  • SSD, ECCV 2016, [paper]
  • Multi Scale CNN, ECCV 2016, [paper]
  • Scale Aware Fast R-CNN, IEEE Transactions on Multimedia, 2018 [paper]

• Methods Based on Feature Pyramids

  • FPN, CVPR 2017, [paper]
  • See feature-level imbalance methods

• Methods Based on Image Pyramids

  • SNIP, CVPR 2018, [paper]
  • SNIPER, NeurIPS 2018, [paper]

• Methods Combining Image and Feature Pyramids

  • Efficient Featurized Image Pyramids, CVPR 2019, [paper]
  • Enriched Feature Guided Refinement Network, ICCV 2019, [paper]
  • Super-Resolution for Small Objects, ICCV 2019, [paper]
  • Scale Aware Trident Network, ICCV 2019, [paper]

2.2. Feature-level Imbalance

• Methods Using Pyramidal Features as a Basis

  • PANet, CVPR 2018, [paper]
  • Libra FPN, CVPR 2019, [paper]
  • Representation Sharing for Fast Object Detector Search and Beyond, ECCV 2020, [paper]
  • Fine-Grained Dynamic Head for Object Detection, NeurIPS 2020, [paper]

• Methods Using Backbone Features as a Basis

  • STDN, CVPR 2018, [paper]
  • Parallel-FPN, ECCV 2018, [paper]
  • Deep Feature Pyramid Reconfiguration, ECCV 2018, [paper]
  • Zoom Out-and-In, IJCV 2019, [paper]
  • Multi-level FPN, AAAI 2019, [paper]
  • NAS-FPN, CVPR 2019, [paper]
  • Auto-FPN, ICCV 2019, [paper]
  • AugFPN, CVPR 2020, [paper]
  • Hit-Detector, CVPR 2020, [paper]
  • Pyramid Convolution, CVPR 2020, [paper]

3. Spatial Imbalance

3.1. Imbalance in Regression Loss

• Lp norm based

  • Smooth L1, ICCV 2015, [paper]
  • Balanced L1, CVPR 2019, [paper]
  • KL Loss, CVPR 2019, [paper]
  • Gradient Harmonizing Mechanism, AAAI 2019, [paper]
  • Dynamic Smooth L1, ECCV 2020, [paper]

• IoU based

  • IoU Loss, ACM IMM 2016, [paper]
  • Bounded IoU Loss, CVPR 2018, [paper]
  • Generalized IoU Loss, CVPR 2019, [paper]
  • Distance IoU Loss, AAAI 2020, [paper]
  • Complete IoU Loss, AAAI 2020, [paper]

• Cross entropy based

  • Offset Bin Classification Network, CVPR 2020, [paper]
  • Bucketing Scheme, ECCV 2020, [paper]
  • Distribution Focal Loss, NeurIPS 2020, [paper]

3.2. IoU Distribution Imbalance

• Cascade R-CNN, CVPR 2018, [paper]
• Hierarchical Shot Detector, ICCV 2019, [paper]
• IoU-uniform R-CNN, arXiv 2019, [paper]
• pRoI Generator, WACV 2020, [paper]

3.3. Object Location Imbalance

• Guided Anchoring, CVPR 2019, [paper]
• FreeAnchor, NeurIPS 2019, [paper]

4. Objective Imbalance

• Task Weighting
• Classification Aware Regression Loss, CVPR 2020, [paper]
• Guided Loss, arXiv 2019, [paper]
• aLRP Loss, NeurIPS 2020, [paper]
• RS Loss, ICCV 2021, [paper]

Contact

Please contact Kemal Öksüz (kemal.oksuz@metu.edu.tr) for your questions about this webpage.