Get and estimate the shape of the head using Dlib and OpenCV with python

This time, I would like to extract the shape of the head using Dlib, OpenCV or display the ellipse as a model.

environment ・ Python 3.7.5 ・ Opencv 4.1.2 · Dlib 19.18.0

I refer to the following sites. →https://cppx.hatenablog.com/entry/2017/12/25/231121 I downloaded the trained model from the following site for the acquisition of facial organs. http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2

The programs that have been made so far are as follows.

head.py


import dlib
import cv2

detector = dlib.get_frontal_face_detector()#Face detector
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")
#↑ Predictor(Specifying the path of the trained model)

def get_center(gray_img):#Find the center of gravity of the image ↓ opencv will do it for you
    moments = cv2.moments(gray_img, False)
    try:
        return int(moments['m10'] / moments['m00']), int(moments['m01'] / moments['m00'])
    except:
       return None
    #Occasionally division by zero occurs, so we are processing it ↑
    #The coordinates obtained with this will be the position of the pupil ↑


def is_close(y0, y1):#Are your eyes closed
    if abs(y0 - y1) < 10:#The arguments are the y-coordinate of the upper eyelid and the y-coordinate of the lower eyelid.
        return True      #Distinguish by the difference between the y-coordinates of the upper and lower eyelids
    return False


def head_point(img,parts):
    heads = [
             parts[1],part[16],
            ]
    
#Get pupil coordinates
def eye_point(img, parts, left=True):
    #Face image and facial organ coordinates, left eye or right eye(True left eye)
    #Get the coordinates of the eyeball ↓
    if left:
        eyes = [
                parts[36],
                min(parts[37], parts[38], key=lambda x: x.y),
                max(parts[40], parts[41], key=lambda x: x.y),
                parts[39],
                ]
    #Since it is in the 36th to 39th, extract those 4 points
    else:
        eyes = [
                parts[42],
                min(parts[43], parts[44], key=lambda x: x.y),
                max(parts[46], parts[47], key=lambda x: x.y),
                parts[45],
                ]
    #At the edge of the eyes (36, 39), the one above the upper eyelid(37 or 38)
    #The one below the lower eyelid(40 or 41)
    #Get the coordinates so that the entire eyeball is included when surrounded by a square
    #The right eye is in the coordinates of the left eye+Get 6 ↑
    org_x = eyes[0].x
    org_y = eyes[1].y
    #↑ Save the coordinates to align the origin with the eyeball
    hrg_x = heads[0].x
    if is_close(org_y, eyes[2].y):#If your eyes are closed, you can't detect your eyes
        return None 
#Find out if your eyes are closed
#↓ Trim the eyeballs and binarize them ↓
    eye = img[org_y:eyes[2].y, org_x:eyes[-1].x]#trimming
    _, eye = cv2.threshold(cv2.cvtColor(eye, cv2.COLOR_RGB2GRAY), 30, 255, cv2.THRESH_BINARY_INV)
#Binarization
    center = get_center(eye)#Find the center of gravity
    if center:
        return center[0] + org_x, center[1] + org_y
    return center


def p(img, parts, eye):#The arguments are face image, facial organ coordinates, and pupil coordinates.(Left, right)
    if eye[0]:#The pupil coordinates are displayed as green dots and the facial organ coordinates are displayed as blue dots on the face image.
        cv2.circle(img, eye[0], 3, (255, 255, 0), -1)
    if eye[1]:
        cv2.circle(img, eye[1], 3, (255, 255, 0), -1)

    for i in parts:
        cv2.circle(img, (i.x, i.y), 3, (255, 0, 0), -1)

    cv2.imshow("face", img)

#Get video from camera
cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()
#Add processing here---

#For example, if you want to get the coordinates of the top of your nose
#parts[33].x
#parts[33].y    

    #↓ Get facial organ coordinates
    dets = detector(frame[:, :, ::-1])
    if len(dets) > 0:
        parts = predictor(frame, dets[0]).parts()

        left_eye = eye_point(frame, parts)#In addition to the facial organs of the eyes, the eyes are also displayed
        right_eye = eye_point(frame, parts, False)

        p(frame, parts, (left_eye,right_eye))#frame*At 0*If you do not 0
#Dots are displayed overlaid on the original face image
#

    if cv2.waitKey(1) == 27:
        break
#Release the capture and close all windows
#So far----
cap.release()
cv2.destroyAllWindows()

#If you don't have a camera
#frame = cv2.imread("Image path")
#cv2.imshow("me", frame) #For confirmation
#Add processing here---

#So far----

#cv2.waitKey(0)
#cv2.destroyAllWindows()


I would like to add more to this program to make a model of the head. I wish I could extract it, but I would like to make an elliptical model (head model) that connects to the 68 landmarks 1 and 17 in the image below. I did a lot of research, but it didn't work, so if anyone can understand it, please let me know. I look forward to working with you. images.jpeg

Recommended Posts

Get and estimate the shape of the head using Dlib and OpenCV with python
Head orientation estimation using Python and OpenCV + dlib
Estimate the attitude of AR markers with Python + OpenCV + drone
Read the graph image with OpenCV and get the coordinates of the final point of the graph
Get the number of articles accessed and likes with Qiita API + Python
Get the stock price of a Japanese company with Python and make a graph
Try projective transformation of images using OpenCV with Python
Visualize the range of interpolation and extrapolation with python
Get the operation status of JR West with Python
[Python] Using OpenCV with Python (Basic)
Using OpenCV with Python @Mac
I tried "gamma correction" of the image with Python + OpenCV
Get rid of dirty data with Python and regular expressions
Sample of HTTP GET and JSON parsing with python of pepper
Play with the password mechanism of GitHub Webhook and Python
Get the source of the page to load infinitely with python.
Environment construction of python and opencv
Shining life with Python and OpenCV
The story of Python and the story of NaN
[Python] Using OpenCV with Python (Image Filtering)
Neural network with OpenCV 3 and Python 3
[Python] Using OpenCV with Python (Image transformation)
[Python] Using OpenCV with Python (Edge Detection)
Get the weather with Python requests
Get the weather with Python requests 2
Coexistence of Python2 and 3 with CircleCI (1.0)
Basic study of OpenCV with Python
I compared the speed of Hash with Topaz, Ruby and Python
Note: Get the first and last items of Python OrderedDict non-destructively
Color extraction with Python + OpenCV solved the mystery of the green background
How to get followers and followers from python using the Mastodon API
[Python] How to get the first and last days of the month
Aligning scanned images of animated video paper using OpenCV and Python
Get the title and delivery date of Yahoo! News in Python
The fastest way to get camera images regularly with python opencv
PhytoMine-I tried to get the genetic information of plants with Python
Using Python and MeCab with Azure Databricks
Check the existence of the file with python
Try using the camera with Python's OpenCV
Capturing images with Pupil, python and OpenCV
Working with OpenStack using the Python SDK
Easy introduction of python3 series and OpenCV3
I'm using tox and Python 3.3 with Travis-CI
I replaced the numerical calculation of Python with Rust and compared the speed
Calculate the shortest route of a graph with Dijkstra's algorithm and Python
How to crop the lower right part of the image with Python OpenCV
Get the number of searches with a regular expression. SeleniumBasic VBA Python
Get images of OpenStreetMap and Geographical Survey Institute maps with Python + py-staticmaps
How to get the date and time difference in seconds with python
I tried to get the authentication code of Qiita API with Python.
Get and convert the current time in the system local timezone with python
Find the general terms of the Tribonacci sequence with linear algebra and Python
I tried to get the movie information of TMDb API with Python
Get the return value of an external shell script (ls) with python3
I measured the speed of list comprehension, for and while with python2.7.
Try to separate the background and moving object of the video with OpenCV
Recognize the contour and direction of a shaped object with OpenCV3 and Python3 (Principal component analysis: PCA, eigenvectors)
I tried to get and analyze the statistical data of the new corona with Python: Data of Johns Hopkins University
Hello World and face detection with OpenCV 4.3 + Python
I tried "smoothing" the image with Python + OpenCV
Get the contents of git diff from python