add evaluation metrics

README.md

@@ -154,6 +154,11 @@ The details of each running scripts are shown in [runDetails.md](doc/runDetails.
 
 * The details of training iPER dataset from scratch are shown in [train.md](./doc/train.md).
 
+### Evaluation
+Run ```./scripts/motion_imitation/evaluate.sh```.
+The details of the evaluation on iPER dataset in [his_evaluators](./thirdparty/his_evaluators/README.md).
+
+
 ## Announcement
 In our paper, the results of LPIPS reported in Table 1, are calculated by **1 – distance score**; 
 thereby, the larger is more similar between two images. The beginning intention of using **1 – distance score** is that it is more accurate to meet the definition of **Similarity** in LPIPS.

thirdparty/his_evaluators/README.md

@@ -154,7 +154,8 @@ For cross-imitation (unpair_types), it provides,
 * `fid`: Frechet Inception Distance (InceptionV3 backbone);
 * `PCB-CS-reid`: the cosine similarity of a pre-trained person re-identification model, 
 PCB [PCB-Net backbone](http://openaccess.thecvf.com/content_ECCV_2018/papers/Yifan_Sun_Beyond_Part_Models_ECCV_2018_paper.pdf).
-* `PCB-freid`: the Frechet Distance of PCB-Net.
+* `PCB-freid`: the Frechet Distance of PCB-Net, and it is very slow, O(n^3). n is the dimension of the feature of PCB-Net,
+and n = 12,288‬.
 * `OS-CS-reid`: the cosine similarity of a pre-trained person re-identification model, 
 [OS-Net](https://github.com/KaiyangZhou/deep-person-reid/blob/master/torchreid/models/osnet.py).
 * `OS-freid`: the Frechet Distance of OS-Net.

thirdparty/his_evaluators/his_evaluators/evaluators/motion_imitation.py

@@ -163,7 +163,7 @@ def __init__(self, dataset, data_dir):
         self.paired_metrics_runner = None
         self.unpaired_metrics_runner = None
 
-    def build_metrics(self, pair_types=("ssim", "psnr", "lps"), unpair_types=("is", "fid", "freid", "reid_score")):
+    def build_metrics(self, pair_types=("ssim", "psnr", "lps"), unpair_types=("is", "fid", "PCB-freid", "PCB-CS-reid")):
         paired_metrics_runner = PairedMetricRunner(metric_types=pair_types)
         unpaired_metrics_runner = UnpairedMetricRunner(metric_types=unpair_types)
 

thirdparty/his_evaluators/his_evaluators/protocols/iPER.py

@@ -276,3 +276,7 @@ def total_frames(self):
             length = len(os.listdir(src_vid))
             total += length
         return total
+
+
+class ICCVIPERProtocol(IPERProtocol):
+    raise NotImplementedError