For more details, please refer to the Master's Thesis.
Framework overview: Data preprocessing steps are (a), (b), and (c). The detection model (a), CenterPoint [60], predicts 3D bounding boxes from a point cloud input. The tracking model (b), ImmortalTracker [55], generates tracks based on these detections, which are denoised (c) based on their lengths. Finally, our anomaly detector (d) identifies anomalous detections within these tracks, leveraging track features and their associated point clouds.
Any questions or suggestions are welcome!
Friedrich Dang friedrich.dang@mail.de
Accurate 3D object detection is critical for autonomous driving, yet manually labeling large-scale LiDAR datasets remains a resource-intensive process. While existing automated labeling methods offer potential solutions, they are often hindered by high computational demands, dependence on extensive labeled data, and limited effectiveness for rare or underrepresented object classes.
To address these challenges, we propose a modular framework that refines object detections and reduces false positives by focusing on anomaly detection. Leveraging a track-centric data structure, our approach filters incorrect predictions while retaining valid ones. Evaluated on the nuScenes dataset, the framework significantly improves detection performance, particularly for rare object classes, in both fully supervised and semi-supervised settings. These results demonstrate the potential of scalable and lightweight frameworks to enhance automated 3D labeling while tackling real-world data constraints in autonomous driving.
Our framework combines CenterPoint for initial detections, ImmortalTracker for track generation, and AnomalyAttention for anomaly refinement. In the semi-supervised 5% seed case, the framework achieves a +6.67% mAP improvement, with notable gains in rare classes like Bus (+9.52%) and Bicycle (+8.31%), illustrating its potential in data-scarce conditions. For the fully supervised 100% seed case, it achieves a +0.87% mAP improvement, effectively reducing false positives while retaining nomalies. These results suggest that lightweight and scalable anomaly detection frameworks can support automated 3D data labeling while addressing real-world data constraints.
- ONCE_Benchmark: Implementation of CenterPoint on the ONCE dataset
- CenterPoint-KITTI: Reimplementation of CenterPoint on the KITTI dataset
- OpenPCDet: Implementation of CenterPoint in OpenPCDet framework (with configs for Waymo/nuScenes dataset)
- AFDet: another work inspired by CenterNet achieves good performance on KITTI/Waymo dataset
- mmdetection3d: CenterPoint in mmdet framework
- CenterPointTensorRT: CenterPoint-PointPillar for accelerated inference with TensorRT
- CenterPoint-ONNX: Convert CenterPoint-Pillar to ONNX / TensorRT
Please refer to INSTALL from the original CenterPoint repo to set up libraries needed for distributed training and sparse convolution. For this project, a complete list of list of all used packages and their versions can be found in our requirement.txt.
Please refer to GETTING_START to prepare the data. Then follow the instruction there to reproduce our results. All detection configurations are included in configs.
AnomalyAttention is developed based on a forked version of CenterPoint. Note that nuScenes' dataset is under non-commercial licenses.
This project is not possible without multiple great opensourced codebases. We list some notable examples below.
Special thanks to SetLabs and TUM's Department for Cyber Physical Systems for providing all necessary ressources, and to Max Ronecker (SetLabs) and Tobias Ladner (TUM) guidance for this project.