Digital Door Lock Control with Face Recognition
Last updated
Last updated
Date: 2021-6-21
Author: Keunho Lee, Yoonkyoung Song
Github: https://github.com/DLIP-Digital-Doorlock-Control-Deep/Digital_Doorlock_Control_Face_Recognition
In this LAB, we started a project to learn human faces and control only authorized users to open digital door locks. After creating an authorized user's dataset to learn about the face, the digital door lock is opened for registered users. To achieve this in an embedded environment, a real-time door lock can be controlled using a Web Cam, Jetson Nano, Arduino UNO, and digital door lock. In addition, the OpenCV library was used to find and learn faces processed by software.
The full version of this project Code is in the next following github URL.
https://github.com/DLIP-Digital-Doorlock-Control-Deep/Digital_Doorlock_Control_Face_Recognition
To build hardware, video cam, Jetson Nano, Arduino UNO, and digital door lock are used. The hardware connection is shown in the figure above. First, we receive frames from where the camera is located through a video cam and search for faces in the image on Jetson Nano. When an authorized user's face is recognized, it sends a command to Arduino via serial communication. Arduino is connected to the digital door lock. When Arduino is ordered to receive signals from Jetson Nano, Arduino command digital door locks to open.
In this project, it is a system that trains the faces of people who have registered in advance (in this project, two pre-registered authorized users), and sends a command to open the door if the similarity with the faces of the people who have learned in advance is higher than the threshold.
Haar Cascade Detection Algorithm is an algorithm that detects a specific object in an image. In this project, it has used to detect human faces in an image.
This algorithm is an algorithm that finds a specific object in an image by learning the pattern for the characteristics of the object. This algorithm is also frequently used for face detection. The algorithm of Haar Cascade is simply described as follows.
First, there are similar features exist in human faces. When we call this a Haar like feature for human face, the key is to recognize it as a face when the threshold is exceeded for each feature.
Adaboost learning algorithm is a simple and efficient classifier to select a small number of critical visual features from a very large set of potential features.
Cascade classifier algorithm configures all cascades for the Haar cascade and detects the face object to be found only when the entire cascade has agreed about the feature.
In OpenCV, it offers "haarcascade_frontalface_default.xml" file, which is pretrained xml file for frontal face.
The face found by Harr Cascade Detection Algorithm, now goes next stage to extract the characteristics of each person by LPBH algorithm
The LBPH algorithm stands for Local Binary Patterns Histogram, which literally means a filter with a specific size is applied to the image. After applying a threshold to the brightness of each pixel, it is classified into binary, and then a histogram is drawn by calculating the pattern between the binaries in the filter. In this way, the human face can be divided into several sections and the histogram value can be calculated in each section. If these histogram values are similar, it is judged as the same person.
Part 1. Creating Authorized Users Face Dataset
Real-time user recognizing code using web cam and save the output video. If you don't have arduino and just want to run this code, you just comment out of the 8 to 11 lines and 77 line, which is related with running the Arduino.
When you want to run in window with arduino, just change the line number 8 to like 'COM3', which corresponds to the arduino port number.
Real-time Face Recognition with embeded digital door lock system
Although the purpose of the digital door lock system is high accuracy recognition of authorized users, however, the most important point in terms of security is to prevent unauthorized persons from being recognized as authorized persons. We raise the confidence threshold in the program so that even if the digital door lock will not be opened every time when authorized person try to open the door lock system,but the unauthorized person is not falsely detected as the authorized user.
The precision, recall, and accuracy from the confusion matrix of this system are as follows.
(Result from the demo videos, checking every single frame by manual. Two demo videos are used.)
Accuracy = 60%
Precision = 100%
Recall = 37.2%
In the case of Accuracy, among all cases of judgment, it is the percentage of all correct answers, which means judging the unauthorized person as an unauthorized person and the authorized person as an authorized person.
In the case of Precision, it is the percentage of actually judged an authorized person out of all cases determined to be an authorized person. If the precision is not high, an unauthorized person may also be mistakenly recognized as an authorized person..
Recall is an indicator of how much a person is recognized to be an authorized person in a situation where they are actually authorized persons. If the recall is not high, the recognition speed will not be fast in the Real-Time system.
The most critical metric from our point of view was Precision. Since even a single false detection can be quite vulnerable to security, we tried to make the precision as close to 100% as possible.
Viola, Paul, and Michael Jones. "Rapid object detection using a boosted cascade of simple features." Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001. Vol. 1. IEEE, 2001.
Viola, Paul, and Michael J. Jones. "Robust real-time face detection." International journal of computer vision 57.2 (2004): 137-154
Haar Cascade Detection Algoritm
https://docs.opencv.org/4.1.0/dc/d88/tutorial_traincascade.html