Face Liveness Detection via OpenCV and Tensorflow

Face Liveness Detection is a biometric system that can identify whether a face comes from a real person or a fake person and prevents the system from giving false verification.

If it comes from a real person, then the face recognition system accepts such images and does the recognition process but it rejects the image if it comes from a fake person. Here, the main idea behind face liveness detection is to enhance the security of face recognition systems.

Table Of Contents

1. Need For Face Liveness Detection
2. Types of Attacks In Face Recognition Systems
   • Print Photo Attack
   • Mobile Photo Attack
   • Replay Attack
   • Photo Mask Attack
   • 3D Mask Attack
3. Solutions For Face AntiSpoofing System
   • Eye Blink Detection
   • Challenge Response System
   • 3D Camera
   • Deep Learning Approach
4. Inspiration For Building Face Liveness Detection
5. Where Face Liveness Detection is used in Face Recognition System?
6. How Face Liveness Detection works?
7. Face Liveness Detection Project Structure
8. Setting Up Our Workspace
9. Building the system
10. Testing the system

Looking for the source code to this post? Get the code on github.

Need For Face Liveness Detection

As Face Recognition System is improving and achieving new heights, attackers are also being smart. They are inventing new techniques to attack the security of such systems. Anyone can trick a face recognition system by using photos, videos, or a different substitute for an authorized person’s face. So, let me clarify this scenario with the help of an example :

Suppose you have a face unlock system on your phone, what if someone can access your phone by showing your photos or videos from their phones. This can be too dangerous and someone can access your personal and confidential information without you even knowing it.

So in this post, we are first going to cover the types of attacks in face recognition systems, then we will look into the solutions to prevent such attacks and finally build a face liveness detection method that can prevent such attacks.

Types Of Attacks In Face Recognition System

Spoofing attacks is still a challenge and a security concern in face recognition systems. Some of the spoofing attacks that can fool a face recognition system security are as follows:

1. Print Photo Attack : Print Photo Attack is a type of attack in which the attacker uses the printed photo of a person’s face to break the face recognition security.

2. Mobile Photo Attack : Mobile Photo Attack is a type of attack in which the attacker tries to attack the system by showing the photos from their mobile phones as a substitue for an authorized person’s face. This type of attack is too common as the digital photo of a person can be easily found in Facebook, Instagram and other social media and the attackers uses such images as a target to bypass the system security.

3. Replay Attack : Replay Attack is a type of attack in which the attacker uses the video of a person’s face as a substitue for an authorized person’s face to attack the system.

4. Photo Mask Attack : In Photo Mask Attack, the attacker create a mask from a 2d photo and uses that mask to spoof or attack the system.

5. 3D Mask Attack : 3D Mask Attack is one of the most sophisticated and expensive attack among all other attacks. In this type of attack, the attacker constructs a  3d face mask of a target client and uses it to gain access to the system.

Here, we have covered all the types of attack that exists in the Face Recognition System. Now, it’s time to look into the solutions that we can adopt to prevent such kinds of attack. The solution to prevent such kinds of attack is Face Liveness Detection.

Solutions For Implementing Face Liveness Detection

1.Eye Blink Detection: Eye Blink Detection can be a relevant solution for Mobile Photo Attack and Print Photo attack because the static images are provided as an input to present such kinds of attack. However, human eyes blink around 15 to 20 times in a minute. So, it would be a reliable solution to prevent such kinds of attacks if we can implement eye blink detection in face recognition systems.

2. Challenge Response System: In Challenge Response System, one entity sends a challenge to another entity and a second entity must respond with an appropriate answer in order to be authenticated. Some of the challenge response questions that can be implemented in the Face Recognition system are asking a user for smiles, expressions of emotions such as sadness, surprise, head movements, and so on. The user/attacker must respond with an answer to be authenticated.

3. 3D Camera: The most reliable anti-spoofing technique uses a 3D camera. If we use 3D camera as an antispoofing solution then we can estimate the depth information of any face image. We can use this depth information to get an estimate of whether it is a real person or not since fake images lack depth value. Precise pixel depth information provides high accuracy against presentation attacks.

4. Deep Learning Appraoch : The deep learning architectures (CNN) have proved that they are very good at tackling image classification problems. So, we can also think of the face liveness detection method as a binary classification problem in which we train our CNN to learn feature maps to clearly distinguish between real image and the spoof image. However, deep learning-based solutions will only be strong within the certain datasets, but the network won’t work in real conditions. For making this solution more viable, the network must be trained with millions of images with different environmental conditions and device specifications so that the network can generalize.

Inspiration For Building Face Liveness Detection

Most face recognition systems are not able to distinguish between the real faces and the spoofed ones and as a result the attackers can easily bypass the system security by showing their pictures to the camera using their phones. So, such systems are not considered reliable enough as they are vulnerable to spoofing attacks.

Due to this reason, there is a strong need to upgrade these systems to make them more secure and reliable. So by observing all these things we are inspired to build this project “Face Liveness Detection” which not only improves the existing system but also adapts to some of the security challenges making this system more secure and reliable. 

Here, we have used the deep learning approach where we have treated the solution as a binary classification problem for identifying ‘real’ or ‘spoof’ faces.

Where Face Liveness Detection Is Used In Face Recognition System ?

After understanding what face antispoofing system is, questions may arise where Face Antispoofing system comes into picture in Face Recognition Systems. We get an answer to this question by looking at the above figure. Liveness detection is done just before face recognition process.

At first, the staff faces towards the webcam and the webcam captures the portrait face of each staff.

Then on those captured portraits, we apply face detection algorithm. After that, the detected face is forwarded to the liveness detection method for checking whether that detected face comes from a live person or not. If it comes from a live person then only the face recognition process is done.

How Face Liveness Detection Works?

Here, we are going to understand how face liveness detection works with the help of a flowchart. As we know, we have two possibilities of inputs as well as outputs in face liveness detection. First possibility is a ‘real’ face and second possibility is a ‘fake’ face.

The portrait face of a person is provided as an input to the webcam and forwarded to the face detection process. Here we use opencv’s haarcascade classifier for detecting the face. After face detection, the detected face is resized into 160 by 160 pixels.

Then those resized faces are provided as an input to the liveness model or the antispoofing model. Finally, the model does the prediction of whether the image comes from a real person or a fake person.

If it comes from a real person, the model predicts the output as ‘real’ and if it come from a fake person, then the model predicts the output as ‘spoof’.

Face Liveness Detection Project Structure

Our system has two top level directories :

1. antispoofing_models/ : contains pretrained model weight file and configuration file for face antispoofing system. ‘antispoofing_model.h5’ is a pretrained model weight file whereas ‘antispoofing_model.json’ is the configuration file of the model.
2. models/ : contains pretrained model for face detection algorithm. Here OpenCV’s ‘haarcascade_frontalface_default.xml’ is used as a face detection model.

We also have two main file in the root directory:

1. livelines_net.py : This is the main program file which opens a webcam and loads antispoofing model to classify whether the image in the webcam comes from a real person or spoof.
2. requirements.txt : This text file contains all the requirements and dependencies file that you need to install before running this system.

Setting Up Our WorkSpace

First, we are going to set up our development environment for this project. We are going to follow some of the steps for setting up our development environment:

pip install opencv==4.2.0.34

pip install tensorflow==2.4.0

import cv2
from tensorflow.keras.preprocessing.image import img_to_array
import os
import numpy as np
from tensorflow.keras.models import model_from_json

Lines 1-5 import our required packages. This script requires OpenCV, Os, Tensorflow and NumPy.

root_dir = os.getcwd()
# Load Face Detection Model
face_cascade = cv2.CascadeClassifier("models/haarcascade_frontalface_default.xml")
# Load Anti-Spoofing Model graph
json_file = open('antispoofing_models/antispoofing_model.json','r')
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json)
# load antispoofing model weights 
model.load_weights('antispoofing_models/antispoofing_model.h5')
print("Model loaded from disk")

video = cv2.VideoCapture(0)
while True:
    try:
        ret,frame = video.read()
        gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
        faces = face_cascade.detectMultiScale(gray,1.3,5)
        for (x,y,w,h) in faces:  
            face = frame[y-5:y+h+5,x-5:x+w+5]
            resized_face = cv2.resize(face,(160,160))
            resized_face = resized_face.astype("float") / 255.0
            # resized_face = img_to_array(resized_face)
            resized_face = np.expand_dims(resized_face, axis=0)
            # pass the face ROI through the trained liveness detector
            # model to determine if the face is "real" or "fake"
            preds = model.predict(resized_face)[0]
            print(preds)
            if preds> 0.5:
                label = 'spoof'
                cv2.putText(frame, label, (x,y - 10),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0,0,255), 2)
                cv2.rectangle(frame, (x, y), (x+w,y+h),
                    (0, 0, 255), 2)
            else:
                label = 'real'
                cv2.putText(frame, label, (x,y - 10),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0,255,0), 2)
                cv2.rectangle(frame, (x, y), (x+w,y+h),
                (0, 255, 0), 2)
        cv2.imshow('frame', frame)
        key = cv2.waitKey(1)
        if key == ord('q'):
            break
    except Exception as e:
        pass
video.release()        
cv2.destroyAllWindows()

In lines 17-24, the webcam is initialized and then the webcam captures the portrait face of a user. On those captured portraits, face detection algorithm is applied to detect and extract the facial region.

In lines 25-52, the extracted face image is resized into 160 by 160 pixels. Then it is forwarded to the antispoofing model for prediction. Then the model does the prediction according to the threshold value. If that value is greater than 0.5 then the predicted class is ‘spoof’ and if the value is lesser than 0.5 then the predicted class is ‘real’. And finally we close our webcam.

Testing The System

Finally, it’s time to test the system. For testing the system, we have to run our main file which is ‘livelines_net.py’. For running this file, open terminal in your current working directory and the use the following command :

python livelines_net.py

After running this file, the web camera got initialized and the system can detect ‘real’ if the real face is provided as an input to the system and ‘spoof’ if we try to use photos or videos to spoof the system.

Conclusion

I hope you were able to follow along and was able to build the system successfully.

Note : The pretrained antispoofing model is not the state of the art solution. It has been trained on few datasets and wrk on some environmental condition. So, there is a room for improving the model performance. For improving the accuracy of the model, you have to train the model with large datasets of images.

To download the source code to this post, Get the code on github.

If you have any questions, recommendations, or critiques, I can be reached via LinkedIn. Feel free to reach out to me.

Check out my Previous Post: Face Recognition Attendance System

FAQ