bpc_na_vinci_code

This is Na Vinci Code's Bin Picking Challenge repository.

Our solution offers a solution to the problem of 6D pose estimation of seen objects.

Table of content

• Datasets

  • IPD

• Evaluation metrics

• Team Members

• Credits

Datasets

The bpc challenge will be ranking the teams based on the mean average Precision (mAP) over a set of MSSD (Maximum Symetry-Aware Surface Distance) thresholds on the IPD dataset.

The dataset is available in the BOP format on HuggingFace.

Method obtained can also be tested on the other datasets availble in the BOP format, such as the T-LESS dataset.

IPD

The IPD dataset is a dataset made by intrinsics.

The training images are available in the train_pbr split, and the validation images are available in the val split. The former is composed of symthetic images and the latter of real images. Both are annotated with the ground truth 6D poses of the objects in the scene in the scene_gt.json file.

The test images are available in the test split, and are not annotated with the ground truth 6D poses of the objects in the scene.

For the training split :

• Scenes 000000-000004 contain objects 0, 8, 18, 19, 20.
• Scenes 000005-000009 contain objects 1, 4, 10, 11, 14.

Every scene is offered from the view of multiple multi-model cameras, including rgb, depth, and polarized images. Masks are also available for the pbr training images, with both the visible part, and the entire object (including the occluded parts).

Here are some examples of the images in the dataset :

• train_pbr split :

modality	Cam 1	Cam 2	Cam3
rgb
depth
aolp
dolp

• val and test splits :

modality	Cam 1	Cam 2	Cam3	photoeno
rgb
depth
aolp
dolp

Evaluation metrics

mAP (mean Average Precision): mAP is calculated across a range of MSSD thresholds (2mm to 20mm, in 2mm increments) for recall. Detections with MSSD below a given threshold are considered successful. The final mAP score is the average across all thresholds. Therefore, higher recall, precision, and pose accuracy contribute to a better mAP score

The challenge will be ranking the teams based on the mean average Precision (mAP) over a set of MSSD (Maximum Symetry-Aware Surface Distance) thresholds on the IPD dataset. MSSD is a metric defined in [1] as being :

$$e_{MSPD} \left( \hat{P}, \bar{P}, S_M, V_M \right) = min_{S \in S_M} max_{x \in V_M} \left| \left| \hat{P}x - \bar{P}Sx \right| \right|_2$$

where:

• $\hat{P}$ is the estimated pose
• $\bar{P}$ is the ground truth pose
• $S_M$ is the set that contaains global symetry transformations of the object model $M$, cf "BOP challenge 2020 on 6D object localization" section 2.3
• $V_M$ is the set of vertices of the object model $M$

Then, scores are ranked based on the mAP over a set of MSSD thresholds, ranging from 2mm to 20mm, in 2mm increments:

$$mAP = \frac{1}{10} \sum_{i=1}^{10} AP_i (th = 2i)$$

where:

• $N$ is the number of MSSD thresholds
• $AP_i$ is the average precision at the $i^{th}$ MSSD threshold

$AP(th)$ is calculated as follows:

$$AP(th) =$$

Team Members

• @Tekexa
• @VincidaB

Contribute

To not add notebook outputs to commits :

pip install nbconvert
git config filter.strip-notebook-output.clean 'jupyter nbconvert --ClearOutputPreprocessor.enabled=True --to=notebook --stdin --stdout --log-level=ERROR'  

create a .gitattributes file at the root of the repo and add :

*.ipynb filter=strip-notebook-output  

Credits

[1]

Hodaň, Tomáš, et al. "BOP challenge 2020 on 6D object localization." Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. Springer International Publishing, 2020.