Question about the mIoU on ScanNet validation set

[linhaojia13]

Hi @DylanWusee ,

Thanks for your contribution!

I tried to reproduce your results on ScanNet with default configurations, and there seems a gap between mine and yours.

If I understand your released code correctly, the default configurations is : input-only xyz, evaluation stride-0.5, use Density with MLP.
Table 5 in your paper shows that mIoU is 61.0 under this configurations. However, my reproduced result is 0.596368.

The gap is not ignorable, how can I get a better result? I use the default batch_size 8, should I modify this param?

My log_evaluate.txt is as below:

Namespace(batch_size=8, dump_dir='dump', gpu=0, model='pointconv_weight_density_n16', model_path='pointconv_scannet_raw/best_model_epoch_500.ckpt', num_point=8192, num_votes=5, ply_path='../../data/scannet/scans', with_rgb=False)
Model restored.
2020-06-08 19:28:12.735647
---- EVALUATION WHOLE SCENE----
eval point avg class IoU: 0.596368
Each Class IoU:::
Class 1 : 0.7410
Class 2 : 0.9473
Class 3 : 0.5330
Class 4 : 0.7170
Class 5 : 0.8281
Class 6 : 0.7131
Class 7 : 0.6537
Class 8 : 0.3413
Class 9 : 0.4584
Class 10 : 0.7243
Class 11 : 0.1304
Class 12 : 0.5907
Class 13 : 0.5174
Class 14 : 0.5421
Class 15 : 0.4015
Class 16 : 0.5021
Class 17 : 0.8202
Class 18 : 0.5903
Class 19 : 0.7632
Class 20 : 0.4123