Test results does not match Table1 in the paper

[ZENGXH]

Hi, thanks for releasing the code.

I tried to run CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_gen_test.sh and get the following numbers:

#JSD:0.049739675370807035
 #{'1-NN-CD-acc': 0.5679012537002563,  # In table 1: 1-NN-CD=75.68 
 # '1-NN-EMD-acc': 0.5827160477638245, # In table 1: 1-NN-EMD=75.06
# 'lgan_cov-CD': 0.3580246865749359,  # In table 1: Cov-CD=46.91
# 'lgan_co -EMD': 0.3580246865749359, # In table 1: Cov-EMD=48.40
# 'lgan_mmd-CD': 0.00066999567206949, # In table 1: MMD-CD=0.000217
# 'lgan_mmd-EMD': 0.04634664207696915,# In table 1: MMD-EMD=0.0324

and other number:

# 'lgan_mmd_smp-CD': 0.00022050135885365307,
# 'lgan_mmd_smp-EMD': 0.0322880782186985}
# '1-NN-EMD-acc_f': 0.3333333432674408, # acc tn / (tn+tp)
# '1-NN-EMD-acc_t': 0.8320987820625305, # acc tp / (tp+fn)
# '1-NN-CD-acc_f': 0.2987654209136963,
# '1-NN-CD-acc_t': 0.8370370268821716,

Here is the table 1 from the paper:

It seems that only the JSD is close to the reported number in the paper ?

[linzq321]

Have you solved the problem yet? I have the same question

[ZENGXH]

It's not solved yet.

[TiankaiHang]

Hi, thanks for releasing the code.

I tried to run CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_gen_test.sh and get the following numbers:

#JSD:0.049739675370807035
 #{'1-NN-CD-acc': 0.5679012537002563,  # In table 1: 1-NN-CD=75.68 
 # '1-NN-EMD-acc': 0.5827160477638245, # In table 1: 1-NN-EMD=75.06
# 'lgan_cov-CD': 0.3580246865749359,  # In table 1: Cov-CD=46.91
# 'lgan_co -EMD': 0.3580246865749359, # In table 1: Cov-EMD=48.40
# 'lgan_mmd-CD': 0.00066999567206949, # In table 1: MMD-CD=0.000217
# 'lgan_mmd-EMD': 0.04634664207696915,# In table 1: MMD-EMD=0.0324

and other number:

# 'lgan_mmd_smp-CD': 0.00022050135885365307,
# 'lgan_mmd_smp-EMD': 0.0322880782186985}
# '1-NN-EMD-acc_f': 0.3333333432674408, # acc tn / (tn+tp)
# '1-NN-EMD-acc_t': 0.8320987820625305, # acc tp / (tp+fn)
# '1-NN-CD-acc_f': 0.2987654209136963,
# '1-NN-CD-acc_t': 0.8370370268821716,

Here is the table 1 from the paper:

It seems that only the JSD is close to the reported number in the paper ?

How long does it take to evaluate? (EMD-CD)

[JohanYe]

Hi, thanks for releasing the code.

I tried to run CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_gen_test.sh and get the following numbers:

#JSD:0.049739675370807035
 #{'1-NN-CD-acc': 0.5679012537002563,  # In table 1: 1-NN-CD=75.68 
 # '1-NN-EMD-acc': 0.5827160477638245, # In table 1: 1-NN-EMD=75.06
# 'lgan_cov-CD': 0.3580246865749359,  # In table 1: Cov-CD=46.91
# 'lgan_co -EMD': 0.3580246865749359, # In table 1: Cov-EMD=48.40
# 'lgan_mmd-CD': 0.00066999567206949, # In table 1: MMD-CD=0.000217
# 'lgan_mmd-EMD': 0.04634664207696915,# In table 1: MMD-EMD=0.0324

and other number:

# 'lgan_mmd_smp-CD': 0.00022050135885365307,
# 'lgan_mmd_smp-EMD': 0.0322880782186985}
# '1-NN-EMD-acc_f': 0.3333333432674408, # acc tn / (tn+tp)
# '1-NN-EMD-acc_t': 0.8320987820625305, # acc tp / (tp+fn)
# '1-NN-CD-acc_f': 0.2987654209136963,
# '1-NN-CD-acc_t': 0.8370370268821716,

Here is the table 1 from the paper:

It seems that only the JSD is close to the reported number in the paper ?

Do you know how the _mmd_smp values differ from the first mmd values?

[zzzzaoqi]

您好，感谢您发布代码。

我试图运行 CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_gen_test.sh并获得以下数字：

#JSD:0.049739675370807035
 #{'1-NN-CD-acc': 0.5679012537002563,  # In table 1: 1-NN-CD=75.68 
 # '1-NN-EMD-acc': 0.5827160477638245, # In table 1: 1-NN-EMD=75.06
# 'lgan_cov-CD': 0.3580246865749359,  # In table 1: Cov-CD=46.91
# 'lgan_co -EMD': 0.3580246865749359, # In table 1: Cov-EMD=48.40
# 'lgan_mmd-CD': 0.00066999567206949, # In table 1: MMD-CD=0.000217
# 'lgan_mmd-EMD': 0.04634664207696915,# In table 1: MMD-EMD=0.0324

和其他号码：

# 'lgan_mmd_smp-CD': 0.00022050135885365307,
# 'lgan_mmd_smp-EMD': 0.0322880782186985}
# '1-NN-EMD-acc_f': 0.3333333432674408, # acc tn / (tn+tp)
# '1-NN-EMD-acc_t': 0.8320987820625305, # acc tp / (tp+fn)
# '1-NN-CD-acc_f': 0.2987654209136963,
# '1-NN-CD-acc_t': 0.8370370268821716,

这是论文中的表 1：

似乎只有 JSD 接近论文中报告的数字？

hello，How long did it take you to evaluate this script

[wanglh300]

您好，感谢您发布代码。
我试图运行 CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_gen_test.sh并获得以下数字：

#JSD:0.049739675370807035
 #{'1-NN-CD-acc': 0.5679012537002563,  # In table 1: 1-NN-CD=75.68 
 # '1-NN-EMD-acc': 0.5827160477638245, # In table 1: 1-NN-EMD=75.06
# 'lgan_cov-CD': 0.3580246865749359,  # In table 1: Cov-CD=46.91
# 'lgan_co -EMD': 0.3580246865749359, # In table 1: Cov-EMD=48.40
# 'lgan_mmd-CD': 0.00066999567206949, # In table 1: MMD-CD=0.000217
# 'lgan_mmd-EMD': 0.04634664207696915,# In table 1: MMD-EMD=0.0324

和其他号码：

# 'lgan_mmd_smp-CD': 0.00022050135885365307,
# 'lgan_mmd_smp-EMD': 0.0322880782186985}
# '1-NN-EMD-acc_f': 0.3333333432674408, # acc tn / (tn+tp)
# '1-NN-EMD-acc_t': 0.8320987820625305, # acc tp / (tp+fn)
# '1-NN-CD-acc_f': 0.2987654209136963,
# '1-NN-CD-acc_t': 0.8370370268821716,

这是论文中的表 1：
似乎只有 JSD 接近论文中报告的数字？

hello，How long did it take you to evaluate this script

As for us , we took about 10 minutes