Skip to content

Inconsistency of the result in GPU and CPU? Will crop affect the result a lot? #1772

Closed
@shuangao

Description

@shuangao

I am trying to see the accuracy of the validation set. However I found the result for standalone validation accuracy is not consistent with the accuracy while training.

Like here, I output the top1 top3 and top5 accuracy (with GPU)

I0121 00:26:24.870959 22427 solver.cpp:247] Iteration 30000, Testing net (#0)
I0121 00:29:49.700218 22427 solver.cpp:298]     Test net output #0: loss = 1.92555 (* 1 = 1.92555 loss)
I0121 00:29:49.847443 22427 solver.cpp:298]     Test net output #1: top 1 accuracy = 0.62146
I0121 00:29:49.847465 22427 solver.cpp:298]     Test net output #2: top 3 accuracy = 0.790921
I0121 00:29:49.847482 22427 solver.cpp:298]     Test net output #3: top 5 accuracy = 0.849442

However, when I want to see the accuracy of the model on iteration 30000 with the same validation set, I found the result is not the same: (CPU and python wrapper)
top1: 0.430361445783
top3: 0.632449799197
top5: 0.721124497992
My Code

net = caffe.Classifier('models/fine_tune.prototxt',
                       'models/fine_tune.caffemodel') // initialize the model with 30000 iteration one
net.set_phase_test()
net.set_mode_cpu()
mean = BinaryProbToNPY('models/food256_mean.binaryproto',256,256,3) // load binaryproto
net.set_mean('data', mean)  
net.set_raw_scale('data', 255)  
net.set_channel_swap('data', (2,1,0))  

path = np.genfromtxt('test/val.txt',usecols=(0),delimiter=' ',dtype=None) // load the validation set path
inputs = [caffe.io.load_image('test/val/'+im_f) for im_f in path]
# Will crop strategy cause such a big difference here? I use the whole image instead of crops.
# I cropped the images in training
scores = net.predict(inputs,oversample = False)
np.save('scores.npy',scores) #save result
# true labels in validation set
true_label = np.genfromtxt('test/val.txt',usecols=(1),delimiter=' ',dtype=None)
top_k = np.asarray([score.argsort()[-1::-1] for score in scores])
# top 1,3,5 accuracy
print top_n_acc(top_k,true_label,1)
print top_n_acc(top_k,true_label,3)
print top_n_acc(top_k,true_label,5)

Here is the output

0.430361445783
0.632449799197
0.721124497992

I am not sure whether this is caused by my crop strategy. If not, Hopefully there is no mistakes in my code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions