MLA Project - Fader Networks

Authors: Adrien PETARD, Robin LEVEQUE, Théo MAGOUDI, Eliot CHRISTON

Group: 11

Date: 2023-12-17

Introduction

This repository contains the code for our project of the Machine Learning course at Sorbonne University. We implemented the Fader Networks model described in the paper Fader Networks: Manipulating Images by Sliding Attributes by Lample et al. (2017).

Repository Structure

• data/: Contains the datasets
  • Anno/ (annotation): contains the annotations of the dataset
  • Eval/ (evaluation): contains the evaluation files of the dataset
  • Img/ (images): contains the images of the dataset, here we chose to use the aligned and cropped images
  • Img_etu/ (etu images): contains the images of the custom dataset made by the contributors of the project
  • Img_lite/ (lite images): contains a smaller version of the images of the dataset
  • Img_processed/ (processed images): contains the processed images of the dataset.
• FaderNetworks_NIPS2017/: The authors' implementation of the Fader Networks model.
• Logs/: Contains the logs of the training and the inference.
• src/ (source): Contains the source code of the project.
  • utils/: Contains the utility functions used in the project.
    • image_processing.py: Contains the functions used to process the data.
    • plots.py: Contains the functions used to plot the results and the losses.
    • train_validation_test_split.py: Contains the function used to split the dataset into train, validation and test sets.
  • AutoEncoder.py: Contains the class used to create the autoencoder.
  • Discriminator.py: Contains the class used to create the discriminator.
  • ImgDataset.py (image dataset): Contains the class used to create the dataset.
  • inference.py: Contains the function used to make the inference.
  • Logger.py: Contains the class used to log the training.
  • network_architectures.py: Contains the neural architectures of the autoencoder and the discriminator.
  • objectives.py: Contains the functions used to compute the objectives, reconstruction objective, discriminator objective and adversarial objective.
  • training_loop.py: Contains the function used to train the model.
• FaderNetworks_paper.pdf: Contains the paper of the Fader Networks model.
• make_inference.py: Contains the script used to make the inference.
• Notebook_Groupe11.ipynb: Contains the notebook used to train and infer the model. A brief presentation of the project and the results is also available in the notebook. Note: Annexes for data exploration, inference on the authors' model and GPU kernel configuration are available in the notebook.
• README.md: Contains the README of the project.
• Report_Groupe11.pdf: Contains the report of the project.
• train.py: Contains the script used to train the model.

User Guide

The following section is a user guide for our FaderNetwork project. It will guide you through the process of training and using the model.

Setup

Environment Libraries

The following libraries are required to run the code:

• Python 3
• PyTorch 2
• Tqdm
• Numpy
• Matplotlib
• Pandas

Dataset

To train or infer the model, you will need to have a folder containing the dataset images and a csv file containing the attributes of the images.

Images

The images must be 256x256 pixels and must be in the RGB format. If you have images in another dimension, you can use the build_Img_processed_folder function in image_processing.py to resize them by executing the script.

build_Img_processed_folder(origin_folder='path/to/origin/folder', destination_folder='path/to/destination/folder')

Attributes

The csv file must have the following format:

description
	attribute_1	attribute_2	...	attribute_n
------------	-------------	-------------	-----	-------------
image_name	value_1	value_2	...	value_n

The first row may contain a description of the attributes, but it is not required.

The first column must contain the name of the image file ending with the extension.

The other columns must contain the values of the attributes for each image.

Training

If you want to train the model, you can use the train script train.py. You will probably need to modify some lines of the script to fit your needs.

# initialize the dataset and the data loader (use get_celeba_dataset_lite() for a smaller dataset)
    dataset = get_celeba_dataset()

The get_celeba_dataset function will return a dataset containing all the images and attributes in the data/Img_processed folder and data/Anno/list_attr_celeba.txt file. If you want to use an other dataset, you can use the ImgDataset class in ImgDataset.py.

from src.ImgDataset import ImgDataset
dataset = ImgDataset(attributes_csv_file='path/to/attributes/file', 
                     img_root_dir='path/to/images/folder', 
                     transform=f_img)

Then specify the split of the dataset for training, validation and test, the batch size and the number of epochs.

Everything about the training is stored in the Logs folder in a folder named with the current date and time. You can find the model weights, the objective evolutions and the images generated at the end of each epoch.

Inference

If you want to use the model to generate images, you can use the make_inference script make_inference.py. You will probably need to modify some lines of the script to fit your needs.

Like training, you will need to specify the dataset but also the model weights to use. To do so, you can indicate the path to the Log folder concerned by the inference.

directory = "Logs/start_2023_12_16_13-56-39_logs/" # Path to the Log folder

Then you can specify the attributes you want to use to generate the images. You can also specify the number of images to generate and the number of images per row in the generated image.

Conclusion

We hope that this guide will help you to use our model. If you have any questions, feel free to contact us.