Closed
Description
I am trying to see the accuracy of the validation set. However I found the result for standalone validation accuracy is not consistent with the accuracy while training.
Like here, I output the top1 top3 and top5 accuracy (with GPU)
I0121 00:26:24.870959 22427 solver.cpp:247] Iteration 30000, Testing net (#0)
I0121 00:29:49.700218 22427 solver.cpp:298] Test net output #0: loss = 1.92555 (* 1 = 1.92555 loss)
I0121 00:29:49.847443 22427 solver.cpp:298] Test net output #1: top 1 accuracy = 0.62146
I0121 00:29:49.847465 22427 solver.cpp:298] Test net output #2: top 3 accuracy = 0.790921
I0121 00:29:49.847482 22427 solver.cpp:298] Test net output #3: top 5 accuracy = 0.849442
However, when I want to see the accuracy of the model on iteration 30000 with the same validation set, I found the result is not the same: (CPU and python wrapper)
top1: 0.430361445783
top3: 0.632449799197
top5: 0.721124497992
My Code
net = caffe.Classifier('models/fine_tune.prototxt',
'models/fine_tune.caffemodel') // initialize the model with 30000 iteration one
net.set_phase_test()
net.set_mode_cpu()
mean = BinaryProbToNPY('models/food256_mean.binaryproto',256,256,3) // load binaryproto
net.set_mean('data', mean)
net.set_raw_scale('data', 255)
net.set_channel_swap('data', (2,1,0))
path = np.genfromtxt('test/val.txt',usecols=(0),delimiter=' ',dtype=None) // load the validation set path
inputs = [caffe.io.load_image('test/val/'+im_f) for im_f in path]
# Will crop strategy cause such a big difference here? I use the whole image instead of crops.
# I cropped the images in training
scores = net.predict(inputs,oversample = False)
np.save('scores.npy',scores) #save result
# true labels in validation set
true_label = np.genfromtxt('test/val.txt',usecols=(1),delimiter=' ',dtype=None)
top_k = np.asarray([score.argsort()[-1::-1] for score in scores])
# top 1,3,5 accuracy
print top_n_acc(top_k,true_label,1)
print top_n_acc(top_k,true_label,3)
print top_n_acc(top_k,true_label,5)
Here is the output
0.430361445783
0.632449799197
0.721124497992
I am not sure whether this is caused by my crop strategy. If not, Hopefully there is no mistakes in my code.
Metadata
Metadata
Assignees
Labels
No labels