Improve FPS of face detector

[duwudi]

FPS of the face detector is poor, but it improved once a face is found as it is then tracked by the dlib tracker.

This is a problem if trying to drive a vehicle whilst also looking for a face, especially if the driving relies on feedback control - the feedback will be slowed down quite a lot if the face detector is running constantly and no faces are detected.

I used the timeit module to show the FPS that can be achieved for just the face detection part. This was not run on a Raspberry Pi, so only the relative difference between the methods is important.

timeit on PC

dlib frontal face detector

import timeit

setup = """
import cv2
import dlib
face_detector = dlib.get_frontal_face_detector()
frame = cv2.imread("face_img.jpg")
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
"""

main = "faceRects = face_detector(gray, 0)"

print(timeit.timeit(stmt=main, setup=setup, number=10000))

Result: 290.3404387
FPS: 34

---

cv haar cascade face detector

import timeit

setup = """
import cv2
face_classifier = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
frame = cv2.imread("face_img.jpg")
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
"""

main = "face_classifier.detectMultiScale(gray, 1.3, 5)"

print(timeit.timeit(stmt=main, setup=setup, number=10000))

Result: 119.2817024
FPS: 84

---

dlib face tracker

This is the tracker which will always run once a face is detected, regardless of the detection method used.

import timeit

setup = """
import cv2
import dlib
face_detector = dlib.get_frontal_face_detector()
frame = cv2.imread("face_img.jpg")
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faceRect = face_detector(gray, 0)[0]
face_tracker = dlib.correlation_tracker()
face_tracker.start_track(frame, faceRect)

"""

main = "peak_to_side_lobe_ratio = face_tracker.update(gray)"

print(timeit.timeit(stmt=main, setup=setup, number=10000))

Result (RGB): 63.7790268
FPS: 158

Result (Greyscale): 55.5197838
FPS: 182

---

Raspberry Pi comparison

Dlib Frontal Face Detector

from imutils.video import VideoStream, FPS
import cv2
import dlib

camera = VideoStream(0).start()

face_detector = dlib.get_frontal_face_detector()

fps = FPS().start()

while True:
    frame = camera.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faceRects = face_detector(gray, 0)

    for faceRect in faceRects:
        x1 = faceRect.left()
        y1 = faceRect.top()
        x2 = faceRect.right()
        y2 = faceRect.bottom()

        cv2.rectangle(frame, (x1, y1), (x2, y2), (255, 0, 0), thickness=2)

    cv2.imshow("Frame", frame)
    fps.update()
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

fps.stop()
print(f"[INFO] Elapsed time: {fps.elapsed():.2f}")
print(f"[INFO] Approx. FPS: {fps.fps():.2f}")
cv2.destroyAllWindows()

FPS: 3.06

OpenCV Haar Cascade

import cv2
from imutils.video import FPS, VideoStream

camera = VideoStream(0, resolution=(640, 480)).start()

face_classifier = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

fps = FPS().start()

while True:
    frame = camera.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_classifier.detectMultiScale(gray, 1.3, 5)

    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x + w, y + h), (255, 0, 0), thickness=2)

    cv2.imshow("Frame", frame)
    fps.update()
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

fps.stop()
print(f"[INFO] Elapsed time: {fps.elapsed():.2f}")
print(f"[INFO] Approx. FPS: {fps.fps():.2f}")
cv2.destroyAllWindows()

FPS: 11.19

---

Not all face detectors are made equal, comparison here. The dlib one shows much better results from my experimentation, but it is very slow on the Raspberry Pi.

If changing to the OpenCV Haar version, we need to check it works with the other dlib tools, such as landmark detection - we can simply run the automated tests for this.