Train from scratch with FDF dataset

[elmira784]

First of all, I should say thanks a lot for a detailed write-up of your amazing work. It is really useful for me
I have some questions. I want to train this model from scratch with the FDF dataset. I download it from the Github that you mentioned:

1. There is no mask dataset in it, should I make it? how?
2. It did not separate data to training and validation, how should I do that?
   As you have told I download the dataset and run the code "python train.py configs/fdf/512.py" but İ got the error, "AssertionError: Did not find landmarks at: data/fdf/train/landmarks.npy". I don't know how should İ solve this problem :(

Whenever you get some time, I appreciate your eyes on this.

[hukkelas]

Thanks, I'm glad you found this project useful!

1. There is no mask dataset in it, should I make it? how?

The masks are generated from the bounding boxes for the face region in the FDF dataset. Take a look at this code which generates it in the dataset loader for pytorch.

2. It did not separate data to training and validation, how should I do that?

The following script will automatically split the dataset into train/validation and format the directory as this repository expects.

import json
import numpy as np
import pathlib
import torch


data_dir = pathlib.Path("data/fdf")
metainfo_path = data_dir.joinpath("fdf_metainfo.json")
with open(metainfo_path) as fp:
    metainfo_path = json.load(fp)

val_dir = data_dir.joinpath("val")
train_dir = data_dir.joinpath("train")

for img_id, info in metainfo_path.items():
    subset = info["category"]
    impath = data_dir.joinpath("images", "128", img_id + ".png")
    if subset == "training":
        subset_dir = data_dir.joinpath("train")
    else:
        assert subset == "validation", subset
        subset_dir = data_dir.joinpath("val")
    new_impath = subset_dir.joinpath("images", "128", img_id + ".png")
    new_impath.parent.mkdir(exist_ok=True, parents=True)
    if impath.is_file():
        impath.rename(new_impath)

# Bounding box and landmarks are stored together, where the 50K last indices correspond to validation
imsizes = ["8", "16", "32", "64", "128"]

for imsize in imsizes:
    bb = np.load(data_dir.joinpath("bounding_box", f"{imsize}.npy"))
    bb_train = bb[:-int(50e3)]
    bb_val = bb[-int(50e3):]
    bb_train_path = data_dir.joinpath("train", "bounding_box", f"{imsize}.npy")
    bb_val_path = data_dir.joinpath("val", "bounding_box", f"{imsize}.npy")
    bb_train_path.parent.mkdir(exist_ok=True, parents=True)
    bb_val_path.parent.mkdir(exist_ok=True, parents=True)
    bb_train_path_th = bb_train_path.parent.joinpath(bb_train_path.stem + ".torch")
    bb_val_path_th = bb_val_path.parent.joinpath(bb_val_path.stem + ".torch")
    np.save(bb_train_path, bb_train)
    np.save(bb_val_path, bb_val)
    torch.save(torch.from_numpy(bb_train), bb_train_path_th)
    torch.save(torch.from_numpy(bb_val), bb_val_path_th)


landmarks = (np.load(data_dir.joinpath("landmarks", "128.npy")) / 128).astype(np.float32)
landmarks_train = landmarks[:-int(50e3)]
landmarks_val = landmarks[-int(50e3):]

np.save(data_dir.joinpath("train", "landmarks.npy"), landmarks_train)
np.save(data_dir.joinpath("val", "landmarks.npy"), landmarks_val)

Let me know if that solves your issue :)

[elmira784]

Thanks for helping me get the materials I needed.
After I split the dataset to train/val I ran the code "python3 -m deep_privacy.dataset.build configs/fdf/base.py"
in order to make the mask dataset but I got the error:
"AssertionError: Did not find file. Looked at: data/fdf/val/bounding_box/128.torch"
should I do something else? :,(

[hukkelas]

I updated the script now. The .torch bounding boxes are just the same array stored as a torch tensor instead of numpy.

It should fix the issue :)

[elmira784]

I updated the script now. The .torch bounding boxes are just the same array stored as a torch tensor instead of numpy.

It should fix the issue :)

Hello again,
I know you are too busy to answer these questions, and I really appreciate your help. I think because I am new in this area I am getting this errors but now I am getting this error :(

Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/elmira/DeepPrivacy/deep_privacy/dataset/build.py", line 79, in
batch = next(iter(dl_train))
File "/home/elmira/DeepPrivacy/deep_privacy/dataset/utils.py", line 102, in next
return self.next()
File "/home/elmira/DeepPrivacy/deep_privacy/dataset/utils.py", line 158, in decorator
batch["img"] = interpolate_image(img, get_transition_value())
TypeError: 'NoneType' object is not callable

2. When I am trying to run "File "/home/elmira/DeepPrivacy/deep_privacy/dataset/utils.py", line 158, in decorator
   batch["img"] = interpolate_image(img, get_transition_value())" without parentheses of "get_transition_value" funvtion I am getting the below error:

Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/elmira/DeepPrivacy/deep_privacy/dataset/build.py", line 79, in
batch = next(iter(dl_train))
File "/home/elmira/DeepPrivacy/deep_privacy/dataset/utils.py", line 102, in next
return self.next()
File "/home/elmira/DeepPrivacy/deep_privacy/dataset/utils.py", line 158, in decorator
batch["img"] = interpolate_image(img, get_transition_value)
File "/home/elmira/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/elmira/DeepPrivacy/deep_privacy/dataset/utils.py", line 138, in interpolate_image
y = torch.nn.functional.avg_pool2d(images * 255) // 1
TypeError: avg_pool2d() missing 1 required positional arguments: "kernel_size"

[hukkelas]

Could you post the exact command you use to run the script?

[elmira784]

Could you post the exact command you use to run the script?

sure
python3 -m deep_privacy.dataset.build configs/fdf/base.py

[hukkelas]

This commit should fix your issue.

Also, I recommend using the config "configs/fdf/512.py" if you're planning to train, as this produces the best results.

[elmira784]

ok, First I ran "python3 -m deep_privacy.dataset.build configs/fdf/512.py" and the result was like this:

2. Then I tried "python3 train.py configs/fdf/512.py" and I got this error:

[hukkelas]

This looks like an issue with recent pytorch versions and NVIDIA Apex, see for example here:
NVIDIA/apex#1303

Note that this repo was trained with Pytorch 1.5, where downgrading Pytorch might remove your issue. To reproduce our environemtn setup, check out our Dockerfile.

[elmira784]

2. python3 train.py configs/fdf/512.py

I tried a lot with pytorch 1.5, Cuda 10.1 but I could not install nvidia-apex in that situatiın, Then I tried with pytorch 1.8.1 and cuda 11.1 but now I got this error :|
Is it because of pytorch version again? :(

2022-08-18 20:16:10,251 [INFO ] Dataset loaded from: data/fdf/val. Number of samples:10000, imsize=128
Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 32768.0
Traceback (most recent call last):
File "train.py", line 21, in
trainer.train()
File "/home/elmira/DeepPrivacy/deep_privacy/engine/trainer.py", line 115, in train
self.train_step()
File "/home/elmira/DeepPrivacy/deep_privacy/engine/trainer.py", line 76, in train_step
to_log = self.loss_optimizer.step(batch)
File "/home/elmira/DeepPrivacy/deep_privacy/modeling/loss/optimizer.py", line 111, in step
losses_d = self.step_D(batch)
File "/home/elmira/DeepPrivacy/deep_privacy/modeling/loss/optimizer.py", line 165, in step_D
log[f"fake_score{i}"] = fake_score.mean().detach()
AttributeError: 'list' object has no attribute 'mean'

[elmira784]

2. python3 train.py configs/fdf/512.py"

I have changed optimizer.py line 165 "log[f"fake_score{i}"] = fake_score.mean().detach()" with
"""
F = torch.stack(fake_score)
log[f"fake_score{i}"] = torch.mean(F).detach()"""
It works now but I am not sure it works correctly or not? :)

[hukkelas]

That fix looks correct :)

[elmira784]

That fix looks correct :)

perfect thanks a lot for your help :)

[hukkelas]

No problem. Let me know if you have any other issues! :)