Find the nose!

Table of contents

• Introduction
• Structure
• Usage
• Preview
• Technologies

Introduction

The goal of this project is to train a neural network responsible for finding the tip of the nose person displayed in the picture. During the process the two architectures were tested:

1. Cascade model with architecture following paper.
2. Small model that uses pre-trained ResNet as a backbone. Both models were trained on modified FK dataset. (To the original dataset, there were added, cropped copies of original pictures. Models trained on extended that way dataset performed significantly better on real world data.)

Structure

• notebooks cotains three notebooks:
  • test.ipynb -> load pre-trained Cascade model and test it on a few pictures
  • model.ipynb -> build, train and compare both models
  • data.ipynb -> load and build dataset
• saved models contains saved pre-trained Cascade model
• models contains functions used to build models
• data contains small sample of preprocessed dataset
• utills contains utilility functions
• sprawozdanie.pdf contains short, written in polish, report describing preprocessing, augmentation technics and used models

Usage

1. Make sure your environment meets requirements listed in requirements.txt. (pip install -r requirements.txt)
2. To test the model on your own pictures save them under pictures directory and change file_name in test.ipynb.
3. You can use pre-trained Cascade model as in notebook test.ipynb.
4. To repeat training download FK dataset and save it as "Facial Keypoints" under data directory. Then run notebook data.ipynb to build dataset. After that you can run model.ipynb to train and evaluate models.

Preview

Raw data

Preprocessing

Augumentation

Cropping

Add noise

Horizontal flip

Predictions on test data

Technologies

• Pandas
• Numpy
• PIL
• Tensorflow
• Keras