[Explanation for beginners] OpenCV face detection mechanism and practice (detect MultiScale)

Overview and environment

I investigated and tried the face detection method of openCV. I will explain for beginners. The following are openCV related articles.

-Install OpenCV 3.3 and Python 3.6 on Windows 10 with Anaconda -Specify parameters in openCV face detection to quickly improve detection accuracy -Face detection from multiple image files with openCV and cut out and saved -Tips for efficiently detecting faces with large numbers of images with openCV

The verification environment is as follows. 00.ArchitectureopenCV.JPG

What you could do with openCV face detection

The face was detected for the person image and surrounded by a square frame. 10.Sample00.jpg

Multiple people are okay. Wow! 10.Sample01.jpg

Detects even funny faces! (The person behind can also be detected) 10.Sample02.jpg

You can also switch from face detection to nose detection. 10.Sample03_Nose.jpg

OpenCV face detection mechanism

I haven't looked at the papers or specific algorithms to find out and confirm them, so the following explanations may be incorrect (I'd appreciate it if you could point them out). This is what the author understood in about half a day. Perhaps watching the video "Viola Jones face detection and tracking explained" will give you a better understanding, but realize the long 1.5 hours. Broke ...

Face detection method overview

Judgment A part of the entire image (read image) is cut out and judged according to various criteria. If it is judged as "not a face" even once, the subsequent judgment is not performed and the judgment device can be applied to a part of the next image. By not making all judgments, the speed of face detection processing is realized (the accuracy of individual judgments is low, but the accuracy is maintained as a whole by stacking many). 20.DetectOverview.JPG

I changed the parameters of the API of openCV so that the cutting process on the left side is easy to understand, and dared to make it falsely detected. Faces are detected in a large number of frames, that is, at least this many frames are used for judgment. You can see that the entire judgment image is cut out and individually applied to the judgment device. 30.Sample01.many.jpg

Judgment method

Harr-like detector overview

Face judgment is performed using a Haar-like detector. The judgment process looks like this. 40.HarrLike.JPG The central part of the nose is bright because it is exposed to light, and the area around the nose is shadowed and darkened, so the above judgment method is effective. Judgment alone is simple and quick, but on the other hand, the judgment accuracy is low. Therefore, making many judgments will improve the accuracy as a whole. There are three main types of Haar-like detectors used for the above judgments. HarrLike

Weakness of judgment method

As a weak point, the detection accuracy drops significantly unless the face is in front. For example, the profile shown below could not be detected. 30.Sample02Angles.jpg

Also, even if I rotated my face, it could not be detected. 30.Sample02.angles.JPG

Python code

I verified it using the following code. It also leaves a mark of trial and error. There are many parts that I just copied from the link posted at the end. The code that can be specified when executing the trained model is written in the article "Specify parameters with face detection of openCV to quickly improve detection accuracy". I will. In addition, the code to read multiple images in the folder and cut out and save the detected face part is the article "Face detection from multiple image files with openCV and cut out and save" See / items / 457737530264572f5a5b).

import cv2

#Classifier directory(Obtained from)
# https://github.com/opencv/opencv/blob/master/data/haarcascades/
# https://github.com/opencv/opencv_contrib/blob/master/modules/face/data/cascades/

cascade_path = "./models/haarcascade_frontalface_default.xml"

#Other model files(reference)
#cascade_path = "./models/haarcascade_frontalface_alt.xml"
#cascade_path = "./models/haarcascade_frontalface_alt2.xml"
#cascade_path = "./models/haarcascade_frontalface_alt_tree.xml"
#cascade_path = "./models/haarcascade_profileface.xml"
#cascade_path = "./models/haarcascade_mcs_nose.xml"

#Files used and I / O directories
image_file = "test.jpg "
image_path = "./inputs/" + image_file
output_path = "./outputs/" + image_file

#For directory confirmation(For when things go wrong)
#import os
#print(os.path.exists(image_path))

#File reading
image = cv2.imread(image_path)

#Grayscale conversion
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

#Acquire the features of the cascade classifier
cascade = cv2.CascadeClassifier(cascade_path)

#Execution of object recognition (face recognition)
#image – CV_8U type matrix. Objects are detected in the images stored here
#objects – A vector whose elements are rectangles. Each rectangle contains the detected object
#scaleFactor – Represents the amount of reduction at each image scale
#minNeighbors – Candidate rectangles must contain at least this number of neighbor rectangles
#flags – This parameter is not used in the new cascade. For older cascades, it has the same meaning as for the cvHaarDetectObjects function.
#minSize – The minimum size an object can take. Objects smaller than this are ignored
facerect = cascade.detectMultiScale(image_gray, scaleFactor=1.1, minNeighbors=2, minSize=(30, 30))

#print(facerect)
color = (255, 255, 255) #White

#When detected
if len(facerect) > 0:
    
    #Create a rectangle that surrounds the detected face
    for rect in facerect:
        cv2.rectangle(image, tuple(rect[0:2]),tuple(rect[0:2]+rect[2:4]), color, thickness=2)

    #Saving recognition results
    cv2.imwrite(output_path, image)

Get the file of the trained model used

Face detection and Nose detection from GitHub I am using the trained model file of modules / face / data / cascades / haarcascade_mcs_nose.xml).

I haven't verified which trained model to use with stackoverflow The article "Types and effects of Cascade Classifiers that can be used with OpenCV" is helpful.

Face detection method "cascade.detectMultiScale"

The method "cascade.detectMultiScale" is used for face detection. As you can see from the Official Site, similar methods "cascade.detectMultiScale2" and " There is cascade.detectMultiScale3 ", but it seems that the basic processing does not change, only the parameters are different. I heard it on stackoverflow. ** Parameters "scaleFactor" and "minNeighbors" are very important for improving detection accuracy. ** The author's trial and error is written down in the article "Tips for efficiently detecting faces in large numbers of images with openCV". I understood it with reference to the following information.

-Try object detection (detectMultiScale) with different parameters (scaleFactor edition) -[Try object detection (detectMultiScale) with different parameters (minNeighbors)] (http://workpiles.com/2015/04/opencv-detectmultiscale-minneighbors/)

Reference link

This time, I will post the site I used for studying as a link.

site comment
Face detection with Haar Cascades Japanese translation of the official tutorial
Principle of face detection that even monkeys can understand There is an easy-to-understand explanation about the principle of face detection
Face cutout and recognition Slide-style face detection with easy-to-understand explanations
OBJECT DETECTION : FACE DETECTION USING HAAR CASCADE CLASSFIERS Detailedexplanationofobjectdetectionwithcode(English)
Heroku +Simple face detection API with OpenCV There is a way to tilt your face to improve accuracy
Python+Face detection with OpenCV-Try the evaluator that comes with OpenCV There are verification results with different models and parameters
python+Try face recognition with OpenCV Copy and paste source of the program

Recommended Posts

[Explanation for beginners] OpenCV face detection mechanism and practice (detect MultiScale)
Implementation and explanation using XGBoost for beginners
[Deep learning] Nogizaka face detection ~ For beginners ~
Hello World and face detection with OpenCV 4.3 + Python
[Explanation for beginners] TensorFlow basic syntax and concept
[Ubuntu] [Python] Face detection comparison between dlib and OpenCV
OpenCV for Python beginners
[Python] Accessing and cropping image pixels using OpenCV (for beginners)
Face detection with Python + OpenCV
Anime face detection with OpenCV
[Explanation for beginners] TensorFlow tutorial MNIST (for beginners)
Face detection from multiple image files with openCV, cut out and save