In the mnist example there should be explicit comments about the default nature of the F.nll_loss which is to average the results over the mini-batch. The other thing that can lead to errors is the len(train_loader.dataset) even though that returns the number of samples in the train dataset if someone was to create a split of train data into train/valid using SubsetRandomSampler then calling again len(train_loader.dataset) will still return you the full number of samples in the train set (which is wrong) without reflecting the correct number of samples since the train set now contains X% less samples which have been kept as validation set.
In the mnist example there should be explicit comments about the default nature of the
F.nll_losswhich is to average the results over the mini-batch. The other thing that can lead to errors is thelen(train_loader.dataset)even though that returns the number of samples in thetrain datasetif someone was to create a split of train data into train/valid usingSubsetRandomSamplerthen calling againlen(train_loader.dataset)will still return you the full number of samples in the train set (which is wrong) without reflecting the correct number of samples since the train set now contains X% less samples which have been kept as validation set.