The baseline about the nonlocal resnet50

[icyzhang0923]

Hi~the top1 val error of your model is 64.15625, and the top1 val error of nonlocal resnet 50 in Xiaolong Wang's CVPR18 paper is 73.3, are there some differents between the two models in the training process?

[gsig]

the top1 val error you are referring to is the 'clip number'. That is, it just reflects the accuracy of classifying a single clip of 32 frames. The 73.3 number in Xiaolong's paper you are referring to, and most of the other numbers in Xiaolong's paper are evaluated by averaging the predictions of 10 such clips across the video, and doing fully convolutional testing on the original size of the image etc. (See https://github.com/facebookresearch/video-nonlocal-net#testing )

Looking the log file for the run_i3d_nlnet_400k_32f.sh baseline in the official nonlocal codebase
( https://dl.fbaipublicfiles.com/video-nonlocal/i3d_nonlocal_32x2_IN_pretrain_400k.log )
we can see that at the end of training a comparable number "best_err" (which is 1 - top1_val) is 38.592 which would be top1_val = 0.61408

2018-03-04 17:08:41.549	json_stats: {"LR": 0.01, "batchSize": 64, "best_err": 38.592233009708735, "best_err5": 17.94093851132686, "bn_momentum": 0.9, "currentIter": 400000, "current_learning_rate": 9.999999747378752e-05, "dataset": "", "epoch": 19.981732280022822, "eval_period": 2000, "momentum": 0.9, "nGPU": 8, "num_classes": 400, "test_err": 39.94741100323625, "test_err5": 18.755056634304207, "train_err": 26.20078125, "train_err5": 10.078125, "train_loss": 1.0473662151130847, "used_gpu_memory": "10045 MiB", "weightDecay": 0.0001}

I pushed a commit the has two new baseline files that do the "video testing" for the pretrained model in this repo:
nonlocal_resnet50_3d_kinetics_test implements averaging of 10 clips in each video, and gets "videotop1" of 68.960, which is closer to the numbers reported in Xiaolong's paper. This is still missing the fully convolutional testing.
nonlocal_resnet50_3d_kinetics_fullyconvtest implements averaging of 10 clips and fully convolutional testing, and gets "videotop1" of 69.4836. You can try experimenting with that one to see if you can get it closer to Xiaolong's paper, the public repo doesn't have any more details about the fully convolutional testing.

Also, I expect these numbers to be slightly lower since I wasn't able to experiment with as many training schedules/hyperparameters, and trained on only 4 gpus. There might also be some tricks missing that could boost the numbers a bit.

Let me know if you manage to improve the numbers further!

Hope that helps,
Gunnar

[icyzhang0923]

Thank you for your reply and the exhaustive explanation. If I manage to improve the numbers, I will tell you soon~