Amodal Intra-class Instance Segmentation: Synthetic Datasets and Benchmark [WACV 2024]

Jiayang Ao · Qiuhong Ke · Krista A. Ehinger

Abstract

Images of realistic scenes often contain intra-class objects that are heavily occluded from each other, making the amodal perception task that requires parsing the occluded parts of the objects challenging. Although important for downstream tasks such as robotic grasping systems, the lack of large-scale amodal datasets with detailed annotations makes it difficult to model intra-class occlusions explicitly. This paper introduces two new amodal datasets for image amodal completion tasks, which contain a total of over 267K images of intra-class occlusion scenarios, annotated with multiple masks, amodal bounding boxes, dual order relations and full appearance for instances and background. We also present a point-supervised scheme with layer priors for amodal instance segmentation specifically designed for intra-class occlusion scenarios. Experiments show that our weakly supervised approach outperforms the SOTA fully supervised methods, while our layer priors design exhibits remarkable performance improvements in the case of intra-class occlusion in both synthetic and real images.

Introduction of the Datasets

This repository has released the datasets (download here) for the paper "Amodal Intra-class Instance Segmentation: Synthetic Dataset and Benchmark" (WACV 2024), which includes:

Intra-AFruit, an intra-class occlusion synthetic dataset containing 10 classes of fruits and vegetables, with 210,000 images for training and 45,000 images for testing.

ACom, an intra-class occlusion synthetic dataset containing 10 classes of other common object categories (anise, biscuit, bottle, candy, conch, donut, fire extinguisher, hammer, hat, and toothpaste), with 10,000 images for training and 2,500 images for testing.

We additionally created two datasets for testing purposes:

InterAMix, which considers occlusion scenes of multiple categories of objects by randomly picking instances among all categories.

A real-world image dataset, of intra-class occlusion scenes containing 102 images.

Example Images from our datasets: Intra-AFruit, InterAMix, ACom, and Real Images (ordered from left to right with two examples per column for each dataset).

Citation

If you find this useful in your work, please consider citing the following reference:

@inproceedings{ao2024amodal,
  title={Amodal Intra-Class Instance Segmentation: Synthetic Datasets and Benchmark},
  author={Ao, Jiayang and Ke, Qiuhong and Ehinger, Krista A},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={281--290},
  year={2024}
}

Contact

If you have any questions regarding this work, please send email to jiayang.ao@student.unimelb.edu.au.