Facial recognition tool

Question

I created a small library and an example application in Python for learning about facial recognition and experimenting with it. Right now though, I am loading a list of file names into memory and then allowing the code to iterate over those names in order to do comparisons or to find the faces.

I think my example and the library is pretty inefficient but I am unsure of how best to handle large amounts of photos using my library. Also, it only works on one photo at a time but my understanding is that the facial recognition library uses all cores and I should only look at one photo at a time.

How can I make my code more efficient?

The original facial recognition library I am using is found at https://github.com/ageitgey/face_recognition

My Library:

import face_recognition
import os, os.path
import ntpath
from PIL import Image

def get_image_directory(path):
  imgs = []
  valid_images = [".jpg",".gif",".png",".tga"]
  for f in os.listdir(path):
    ext = os.path.splitext(f)[1]
    if ext.lower() not in valid_images:
      continue
    imgs.append(os.path.join(path,f))
  return imgs

def recognize_faces(path):
  file_name = path_leaf(path)
  image = face_recognition.load_image_file(path)
  face_locations = face_recognition.face_locations(image)
  i = 0
  for face_location in face_locations:
    top, right, bottom, left = face_location
    face_image = image[top:bottom, left:right]
    pil_image = Image.fromarray(face_image)
    pil_image.save( 'output/' + str(file_name) + '_' +str(i)+".png")
    i+=1

def compare_faces(unknown_images, known_images):
  for image in unknown_images:
    unknown_comparison_image = create_face_comparison_encoding(image)
    for k_image in known_images:
      results = face_recognition.compare_faces([k_image],unknown_comparison_image)
      if results[0] == True:
        print("This person appears familiar!")
        print("Writing the faces to output folder!")
        recognize_faces(image)
      else:
        print("This doesn't appear to be familiar!")

def path_leaf(path):
  head, tail = ntpath.split(path)
  return tail or ntpath.basename(head)

def create_face_comparison_encoding(image):
  known_image = face_recognition.load_image_file(image)
  known_image_encoding = face_recognition.face_encodings(known_image)[0]
  return known_image_encoding

Application:

from ml_face import *

##
# The Program
##
unknown_persons = get_image_directory('images/') # Unknown Persons Directory
known_persons = get_image_directory('known_person/') # Known Persons Directory
known_list = [] # Empty List
for k_per in known_persons:
  known_comparison_image = create_face_comparison_encoding(k_per)
  known_list.append(known_comparison_image)
compare_faces(unknown_persons,known_list)

Answer 1

I noticed that the face_recognition library itself is quite optimized to run on both multiple cpus and gpu as well. But I will try to give my two cents for your specific code here.

Typically, there are two steps when it comes to such applications.

There is the training step where you parse all the known_person images once, and build a data of face_encodings, and store them on the disk. Typically these encodings are much smaller than the image data itself in size and it is something you do once for all the known images. (if you add new known images, consider this as a recalibration step where you need to regenerate your encodings). This step is usually accepted to be time-consuming and not done very frequently.

Then there is the inference step, which is the compare_faces function. This is what you want to do several times. For this specific step you just need the new image and known_list(which is the trained data that you stored on disk ahead of time).

So to improve your library, I would suggest adding a 'train' function which allows a list of images. This list should then store the encodings in a format (compressed if you have several images).

Then the compare_faces function can be modified to take in the encodings data. You can split this into load step (in case you compressed the data) and then compare step. This allows application to load the data once, but infer several times.

In your code itself, the compare_faces can be improved already. It can be changed to:

def compare_faces(unknown_images, image_encodings_data):
  for image in unknown_images:
    unknown_data = create_face_comparison_encoding(image)
    # Note how compare_faces takes multiple encodings.
    results = face_recognition.compare_faces(image_encodings_data,unknown_data)
    if results.count(True) > 0:
      print("This person appears familiar!")
      print("Writing the faces to output folder!")
      recognize_faces(image)
    else:
      print("This doesn't appear to be familiar!")

Also, I find it strange that you do not return which face it actually matched to.

Please feel free to ignore me if this is no longer your concern :)