NaN loss when trainign 3-class detectors on waymo datast.

[Eaphan]

Thanks for your sharing. When I tried to train the RDIoU for 3 classes on Waymo dataset, I met the error of NaN loss.
/data/yfzhang/workspace/OpenPCDet/pcdet/models/dense_heads/anchor_head_rdiou_3cat.py(304)
if rpn_loss.isnan(): print(cls_loss) print(box_loss) pdb.set_trace()
Do you have any suggestions? Or have you ever tried to train RDIoU for the 3 classes? How about the performance?
Thanks for your reply in advance.

[hlsheng1]

Hi, changing the clamp max value for l,w,h can solve this issue.

Like l1, w1, h1 = torch.clamp(l1, max=3), torch.clamp(w1, max=3), torch.clamp(h1, max=3)

[hlsheng1]

Hi, the clamp operation must be conducted before exp operation.