Don't overfitting model

Good day!

I tried to overfit model `faster_rcnn_r50_fpn_1x` on my own dataset with 1000 images.
What I did:
  - set flip_rate = 0 (all augmentation off)
  - set weight_decay = 0

I trained model for 20 epoch, but TPR for some classes very small, especially the number of which is small.
Do you use any other regularization?

Thank you for your work