DeepFace_Video_Preparation_Pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from deepface import DeepFace\n",
    "import pandas as pd\n",
    "import cv2\n",
    "import numpy as np\n",
    "import os\n",
    "import matplotlib as plt\n",
    "import math\n",
    "import glob\n",
    "import time \n",
    "#I've added timers at the most computationally expensive places so that you can gauge how much detail is feasible for you to achieve depending on your hardware"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Splitting the relevant videos into frames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**In here, you can set:**\n",
    "* the frames/second rate\n",
    "* the input folder\n",
    "* the output folder\n",
    "* the naming system you want for your output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#For some reason he doesn't like backslashes when specifying file paths so make sure you use frontslashes and put an 'r' infront of the path\n",
    "#Also doesn't like it if your path is not on your local machine (e.g. a OneDrive file)\n",
    "\n",
    "# #This line starts a timer\n",
    "# start = time.time()\n",
    "\n",
    "pathOut = r'C:/Users/chazzers/Desktop/Recordings/Frames/'\n",
    "count = 0\n",
    "counter = 1\n",
    "listing = os.listdir(r'C:/Users/chazzers/Desktop/Recordings/Charles')\n",
    "for vid in listing:\n",
    "    vid = r'C:/Users/chazzers/Desktop/Recordings/Charles/' + vid\n",
    "    cap = cv2.VideoCapture(vid)\n",
    "    count = 0\n",
    "    counter += 1\n",
    "    success = True\n",
    "    while success:\n",
    "        success,image = cap.read()\n",
    "#         print('read a new frame:',success)\n",
    "        if count%1000 == 0 :\n",
    "              cv2.imwrite(pathOut + 'video%d'%counter + 'frame%d.jpg'%count ,image)\n",
    "        count+=1\n",
    "\n",
    "#change the number after modulo sign to get the desired frame rate\n",
    "\n",
    "# #This line stops the timer\n",
    "# elapsed_time_split = (time.time() - start) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dividing the total elapsed time by the number images split to get the time to extract one frame from the video"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "splitsec = elapsed_time_split/count\n",
    "print(splitsec)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Use the code below to split videos into *ALL* its frames -> Not deemed necessary unless one wants to start looking at mircoexpressions. Currently 2 images per second is more than sufficient to provide results in line with external measures of engagement/ affect"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# #splitting the video into individual frames\n",
    "\n",
    "# # Playing video from file:\n",
    "# cap = cv2.VideoCapture(test_video)\n",
    "\n",
    "# try:\n",
    "#     if not os.path.exists('video_frames'):\n",
    "#         os.makedirs('video_frames')\n",
    "# except OSError:\n",
    "#     print ('Error: Creating directory of data')\n",
    "\n",
    "# currentFrame = 0\n",
    "# while(True):\n",
    "#     # Capture frame-by-frame\n",
    "#     ret, frame = cap.read()\n",
    "\n",
    "#     # Saves image of the current frame in jpg file\n",
    "#     name = './test_videodata/frame' + str(currentFrame) + '.jpg'\n",
    "#     print ('Creating...' + name)\n",
    "#     cv2.imwrite(name, frame)\n",
    "\n",
    "#     # To stop duplicate images\n",
    "#     currentFrame += 1\n",
    "\n",
    "# # When everything done, release the capture\n",
    "# cap.release()\n",
    "# cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyzing frames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Change the filename to include the name of the relevant video below\n",
    "* i.e. r'C:/Users/chazzers/Desktop/Recordings/Frames/video2.jpg' if you want video 2 \n",
    "* or r'C:/Users/chazzers/Desktop/Recordings/Frames/video3.jpg' if you want video 3 etc.\n",
    "* or r'C:/Users/chazzers/Desktop/Recordings/Frames/.jpg' if you want all the videos in one go."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Putting image frames into an array so that DeepFace can run on them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # #This line starts a timer\n",
    "# # start = time.time()\n",
    "\n",
    "img_array2 = []\n",
    "for filename in glob.glob(r'C:/Users/chazzers/Desktop/Recordings/Frames/video2*.jpg'):\n",
    "    img = cv2.imread(filename)\n",
    "    height, width, layers = img.shape\n",
    "    size = (width,height)\n",
    "    img_array2.append(img)\n",
    "    \n",
    "# # #This line stops the timer    \n",
    "# # elapsed_time_fl = (time.time() - start) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dividing the total elapsed time by the number images appended to the array to get the time to extract append one image to the array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# arraysec = elapsed_time_fl/len(img_array)\n",
    "# print(arraysec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Analyzing: 100%|███████████████████████████████████████████████████████████████████| 1875/1875 [08:38<00:00,  3.62it/s]\n"
     ]
    }
   ],
   "source": [
    "#Running deepface on the array just created\n",
    "df_test2 = DeepFace.analyze(img_path=img_array2, actions = ['emotion'],enforce_detection=False)\n",
    "# Alos possible to do 'age', 'gender', 'race' but increases the amount of time taken massively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# This returns a python dictionary of emotions\n",
    "#unhash to check\n",
    "# df_test2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inserting the data into a Pandas dataframe so we can manipulate it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = df_test2\n",
    "\n",
    "emotions = set()\n",
    "# First we need to find out all unique emotions\n",
    "for key, value in data.items():\n",
    "    for emotion in value['emotion'].keys():\n",
    "        emotions.add(emotion)\n",
    "\n",
    "rows = []\n",
    "columns = ['instance'] + list(emotions)\n",
    "\n",
    "for key, value in data.items():\n",
    "    rows.append([0] * len(columns)) # Start creating a new row with zeros\n",
    "\n",
    "    key = key.split('_')[1] # Get the 1,2,3 out of the instance\n",
    "    rows[-1][0] = key\n",
    "    for emotion, emotion_value in value['emotion'].items():\n",
    "        rows[-1][columns.index(emotion)] = emotion_value # place the emotion in the correct index\n",
    "\n",
    "\n",
    "df = pd.DataFrame(rows, columns=columns)\n",
    "df.set_index('instance', inplace=True)\n",
    "\n",
    "# print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Saving it to a csv file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(not strictly necessary, but easier to share data later on with excel users)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tkinter as tk\n",
    "from tkinter import filedialog"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting the file path to which the excel file can be exported"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "export_file_path = r'C:/Users/chazzers/Desktop/Recordings/Results'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "root= tk.Tk()\n",
    "\n",
    "canvas1 = tk.Canvas()\n",
    "canvas1.pack()\n",
    "\n",
    "def exportCSV ():\n",
    "    global df\n",
    "    \n",
    "    export_file_path = filedialog.asksaveasfilename(defaultextension='.csv')\n",
    "    df.to_csv (export_file_path, index = True, header=True)\n",
    "\n",
    "saveAsButton_CSV = tk.Button(text='Export CSV', command=exportCSV, bg='green', fg='white', font=('helvetica', 12, 'bold'))\n",
    "canvas1.create_window(150, 150, window=saveAsButton_CSV)\n",
    "\n",
    "root.mainloop()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}