Generalisable and Distinctive (GeDi) 3D local deep descriptors for point cloud registration

3DMatch ⟶ ETH	3DMatch ⟶ KITTI	KITTI ⟶ 3DMatch

Description

An effective 3D descriptor should be invariant to different geometric transformations, such as scale and rotation, repeatable in the case of occlusions and clutter, and generalisable in different contexts when data is captured with different sensors. GeDi is a simple but yet effective method to learn generalisable and distinctive 3D local descriptors that can be used to register point clouds captured in different contexts with different sensors. Point cloud patches are extracted, canonicalised with respect to their local reference frame, and encoded into scale and rotation-invariant compact descriptors by a PointNet++-based deep neural network. Our descriptors can effectively generalise across different sensor modalities from locally and randomly sampled points. The graphs above show the comparison between our descriptors and state-of-the-art descriptors on several indoor and outdoor datasets reconstructed using both RGBD sensors and laser scanners. In particular, 3DMatch is an indoor dataset captured with RGBD sensors, ETH and KITTI are outdoor datasets captured with laser scanners. Our descriptors outperform most recent descriptors by a large margin in terms of generalisation, and become the state of the art also in benchmarks where training and testing are performed in same scenarios.

Paper (pdf)

References

• FPFH: R. Rusu, N. Blodow, and M. Beetz, Fast point feature histograms for 3D registration, in Proc. of IEEE ICRA, Kobe, JP, May 2009 (pdf)
• SHOT: S. Salti, F. Tombari, and L. Di Stefano, SHOT: Unique signaturesof histograms for surface and texture description, CVIU, vol. 125,pp. 251–264, Aug 2014 (pdf)
• 3DMatch: A. Zeng, S. Song, M. Niessner, M. Fisher, J. Xiao, and T. Funkhouser, 3DMatch: Learning the matching of local 3D geometry in range scans, in Proc. of IEEE CVPR, Hawaii, US, Jul 2017 (pdf)
• CGF: M. Khoury, Q.-Y. Zhou, and V. Koltun, Learning compact geometric features, in Proc. of IEEE ICCV, Venice, IT, Oct 2017 (pdf)
• PerfectMatch: Z. Gojcic, C. Zhou, J. Wegner, and W. Andreas, The perfect match: 3D point cloud matching with smoothed densities, in Proc. of IEEE CVPR, Long Beach, US, Jun 2019 (pdf)
• FCGF: C. Choy, J. Park, and V. Koltun, Fully Convolutional Geometric Features, in Proc. of IEEE ICCV, Seoul, KR, Oct 2019 (pdf)
• D3Feat: X. Bai, Z. Luo, L. Zhou, H. Fu, L. Quan, and C.-L. Tai, D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features, in Proc. of IEEE CVPR, Virtual, Jun 2020 (pdf)
• LMVD: L. Li, S. Zhu, H. Fu, P. Tan, and C.-L. Tai, End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds, in Proc. of IEEE CVPR, Virtual, Jun 2020 (pdf)
• SpinNet: S. Ao, Q. Hu, B. Yang, A. Markham, and Y. Guo, SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration, in Proc. of IEEE CVPR, Virtual, Jun 2021 (pdf)
• DIP: F. Poiesi and D. Boscaini, Distinctive 3D local deep descriptors, in Proc. of IEEE ICPR, Virtual, Jan 2021 (pdf)