I want to check the position of my face with OpenCV!

Introduction

When running a program that outputs what you input, such as a game "I wish I could recognize the face and operate it?" I was thinking about it. So, I created a program that uses OpenCV to react the position of the face and find the center point of the face from there. However, even if I think so, it is meaningless if I can not detect it properly with a still image before the program that processes the video, so I made a program that can process the photo and then made it possible to handle it with the video.

Creation overview

Specify the position of the face using OpenCV

Development environment

MacBookPro 2019 2.4GHz core i5 memory 16GB OS:macOS Mojave10.14.6 Python(anaconda)

Contents

The face is captured using the OpenCV library, and the coordinate points of the photo are output.

Read image

For the time being, I created a library that exists in the first place to specify a face without thinking about anything.

Face_photo.ipynb

import cv2
import matplotlib.pyplot as plt

image_name = input("File Name?")

src=cv2.imread(image_name)
image_gray=cv2.cvtColor(src,cv2.COLOR_BGR2GRAY) #Grayscale
color=(0,0,255) #Kakomu guy(Red)
color_c=(255,0,0) #The one who hits(point,Blue)
cascade = cv2.CascadeClassifier("haarcascade_frontalface_alt2.xml") #Import training data

#Find a face
facerect = cascade.detectMultiScale(image_gray, scaleFactor=1.1, minNeighbors=1, minSize=(50,50))

if len(facerect) > 0:
    for rect in facerect:
        cv2.rectangle(src,tuple(rect[0:2]),tuple(rect[0:2]+rect[2:4]),color,thickness = 2) #Lines around the face
        cv2.circle(src,tuple(rect[0:2]+rect[2:4]//2),5,color_c,thickness=-10) #Dot in the center
        print(rect[0:2]+rect[2:4]//2)#Output the center point

cv2.imwrite(image_name+"_result.jpg ",src)#The one to save

This allows you to plot dots on your face. As a result, it looks like this. (Original photo) (Output result)

I think I was able to capture it in a good way.

problem

However, there were the following problems with this method. If you look at the hole in your nose as your eyes and the bottom of your nose as your nose, it's your face (Kippari)

I can't say anything, so I need to fix this as well.

Judge the biggest face

So, this time, let's think about it with the body that we should recognize one person. The reason is that the ultimate goal is to operate with the face, so there is no need to assume many people. So, let's recognize only the big face.

Face_Photo.ipynb

import cv2
import matplotlib.pyplot as plt

image_name = input("File Name?")

# image_name = "27708.jpg " #Import photos

src=cv2.imread(image_name)
image_gray=cv2.cvtColor(src,cv2.COLOR_BGR2GRAY) #Grayscale
color=(0,0,255) #Kakomu guy(Red)
color_c=(255,0,0) #The one who hits(point,Blue)

big_face=(0,0,0,0)

cascade = cv2.CascadeClassifier("haarcascade_frontalface_alt2.xml") #Import training data

#A spell to manage
facerect = cascade.detectMultiScale(image_gray, scaleFactor=1.1, minNeighbors=1, minSize=(50,50))

if len(facerect) > 0:
    for rect in facerect:
        print(rect)
        if(big_face[2]<rect[2]):
            big_face= rect

cv2.rectangle(src,tuple(big_face[0:2]),tuple(big_face[0:2]+big_face[2:4]),color,thickness = 2) #Lines around the face
cv2.circle(src,tuple(big_face[0:2]+big_face[2:4]//2),5,color_c,thickness=-10) #Dot in the center
print("result:%d" + (big_face[0:2]+big_face[2:4]//2))#Output the center point

cv2.imwrite(image_name+"_result.jpg ",src)#The one to save

I decided to save the biggest face in something called big_face. It sounds like a bad word, but it's not. Of course, this is now a good way to save your biggest face.

Recognize with video

At this point, it is the finishing stage. I made it work with video.

Face_Video.ipynb

import cv2
import matplotlib.pyplot as plt

# image_name = input("File Name?")

# image_name = "27708.jpg " #Import photos

cap=cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH,640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT,480)

while True:
    ret,src=cap.read()

    if not ret:
        print("Couldn't detect camera.")
        break

    key=cv2.waitKey(1)&0xff
    if key==ord("q"):
        print("Given a exit command.")
        break
        
    image_gray=cv2.cvtColor(src,cv2.COLOR_BGR2GRAY) #Grayscale
    color=(0,0,255) #Kakomu guy(Red)
    color_c=(255,0,0) #The one who hits(point,Blue)

    big_face=(0,0,0,0)

    cascade = cv2.CascadeClassifier("haarcascade_frontalface_alt2.xml") #Import training data

    #A spell to manage
    facerect = cascade.detectMultiScale(image_gray, scaleFactor=1.1, minNeighbors=1, minSize=(50,50))

    if len(facerect) > 0:
        for rect in facerect:
            if(big_face[2]<rect[2]):
                big_face= rect

        cv2.rectangle(src,tuple(big_face[0:2]),tuple(big_face[0:2]+big_face[2:4]),color,thickness = 2) #Lines around the face
        cv2.circle(src,tuple(big_face[0:2]+big_face[2:4]//2),5,color_c,thickness=-10) #Dot in the center
        print(big_face[0:2]+big_face[2:4]//2)#Output the center point

    cv2.imshow("result",src)
    cv2.waitKey(1)
    
cv2.destroyAllWindows()
cap.release()

If you can do it so far, it will look like the one below. Now it works properly. In addition, the position of the nose itself here can also be output as a numerical value, so if you use this, you will be able to operate at the position of the face. By the way, I put in that it ends when I press q, but it did not respond with jupyter. sorrow.

problem

However, there is a problem with this method, and it does not respond if it is a profile. It does not mean that the profile is the Shinkansen, but there is a problem that it does not respond because it is the data learned using the front face in the first place. So, if you have a camera and a screen completely in front of you and use it in a game, I think you can make full use of it.

That's it for face capture with OpenCV.

Supplement

For all the images used this time, I used the images of Pakutaso.