How to make a face image data set used in machine learning (3: Face image generation from candidate images Part 1)

Previous, Last time Generate candidate images with the target face. I mentioned how to get it, but this time I will introduce how to extract the face area part from them and generate a face image.

Development environment used this time

Flow to generate face image from candidate image

To generate a face image, trim only the area where the face is reflected from the candidate images, and generate the trimmed area as a new image = face image. However, it takes a lot of time and effort to do this work manually, so we will use the following face recognition API (library) to solve it even a little.

If you use these APIs, it is possible to automatically extract only the face area from the candidate image, but this time we will explain the method using OpenCV and dlib. The specific flow of face image generation is basically the same, as follows.

  1. By inputting a candidate image into the face detection API, the coordinate information that seems to be the face area is acquired.
  2. Trim the coordinate area based on the coordinate information obtained in 1 from the candidate image.
  3. Generate the trimmed area as a new image and use it as a face image.

The above process shall be defined by the face_detect method in the code below.

# -*- coding: utf-8 -*-
import glob

#Directory path containing candidate images
IMG_PATH = "/home/hoge/image/"

def face_detect(img_list):
	#Define face image generation process with this method
	return


if __name__ == '__main__':
	
	#Get a candidate image from the specified directory and generate a face image using API etc.
	images = glob.glob(IMG_PATH + '*.jpg')	
	face_detect(images)
	

dlib dlib is a library that summarizes image processing and machine learning functions developed for C ++, but it can also be used with python. (For how to install dlib in Mac environment, please refer to this article etc.)

dlib provides a get_frontal_face_detector method for face detection, so use that. In addition, if you use SVM-related methods, which are a type of non-linear classifier with high discrimination accuracy, more advanced detection is possible, but this time we will prioritize ease of use. The face image generation code that actually uses the get_frontal_face_detector method is as follows.

import dlib
import cv2

def face_detect(img_list):

	#Identifyer call to detect face area
	detector = dlib.get_frontal_face_detector()
	
	#Get coordinate information by applying the candidate image to the classifier
	for img_path in img_list:
	
		#Candidate image for trimming (color mode)
		img = cv2.imread(img_path, cv2.IMREAD_COLOR)
		
		#Reassemble the image array and input it to the classifier to acquire coordinate information that seems to be the face area.
		cv_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
		faces = detector(cv_img, 1)

		#Face image generation if face area is detected
		img_count = 1
        for face in faces:
        	
        	#Get candidate image size
        	height, width = img.shape[:2]
        	
        	#Get the coordinate points of the face area
        	top = face.top()
        	bottom = face.bottom()
        	left = face.left()
        	right = face.right()
        	
        	#Ignore irregular facial areas
        	if not top < 0 and left < 0 and bottom > height and right > width:
            	break
            else:
            	#Face area trimming
            	dst_img = img[top:bottom, left:right]
            	
            	#Face image size Normalized and saved
            	face_img = cv2.resize(dst_img, (64,64))
            	new_img_name = str(img_count) + '.jpg'
            	cv2.imwrite(new_img_name, face_img)
            	img_count += 1

OpenCV and dlib handle different pixel values of RGB (Red / Green / Blue), which is the color information of the image. In OpenCV, RGB (Red / Green / Blue) pixel values are stored in the array in the order of BGR, but in dlib, they are stored in the order of RGB, so it is necessary to convert the array structure using the cvtColor method. There is.

In addition, get_frontal_face_detector rarely returns the coordinate values outside the image area as the coordinate information of the face area. This is because the face is reflected at the edge of the image, so a part of the face is often missing, and it is difficult to use it as learning data as it is as a face image, so ignore it as an irregular face area. I have. OpenCV OpenCV is an image processing library that can be used in C ++, Python, etc., and is familiar in this article series, but here the CascadeClassifier class provided as a cascade classifier class To use. By using this class, it is possible to build a classifier corresponding to various objects depending on the contents of the xml file that stores the learning results to be read when building the classifier. This time, by referring to haarcascade_frontalface_default.xml (provided site) provided by OpenCV, as a face detector I will use it.

import cv2

#XML file path referenced by the HaarCascade classifier
XML_PATH = "./haarcascade_frontalface_default.xml"
    
def face_detect(img_list):   
    
    #Set face area classifier
    classifier = cv2.CascadeClassifier(xml_path)

	#Face image generation if face area is detected
    img_count = 1
    for img_path in img_list:
    
		#Color image for trimming
		org_img = cv2.imread(img_path, cv2.IMREAD_COLOR)

		#Grayscale image for classifier input
        gray_img = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)
        
        #Input the candidate image (grayscale image) into the classifier and acquire the coordinate information that seems to be the face area.
        face_points = classifier.detectMultiScale(gray_img, \
                scaleFactor=1.2, minNeighbors=2, minSize=(1,1))
        
        #A face image is generated by trimming a color image from the identification result (rectangular coordinate information).
        for points in face_points:
            
            #Get the coordinate points of the face area
            x, y, width, height =  points
            
            #Face area trimming
            dst_img = org_img[y:y+height, x:x+width]
            
            #Face image size Normalized and saved
            face_img = cv2.resize(dst_img, (64,64))
            new_img_name = str(img_count) + '.jpg'
            cv2.imwrite(new_img_name, face_img)
            img_count += 1
            

In the method using OpenCV, two types of candidate images are prepared: a color image for trimming and a grayscale image for classifier input. This is because the detectMultiScale method does not support input of color images.

The detectMultiScale method takes the following parameter arguments.

Try object detection (detectMultiScale) with different parameters (minNeighbors) Try object detection (detectMultiScale) with different parameters (scaleFactor edition)

A rough impression of using the above two methods

dlib

OpenCV

This is simple, but this time I explained how to generate a face image from a candidate image.

Recommended Posts

How to make a face image data set used in machine learning (3: Face image generation from candidate images Part 1)
How to make a face image data set used in machine learning (2: Frame analysis of video to obtain candidate images)
How to create a face image data set used in machine learning (1: Acquire candidate images using WebAPI service)
[Part 1] Use Deep Learning to forecast the weather from weather images
[Part 3] Use Deep Learning to forecast the weather from weather images
[Part 2] Use Deep Learning to forecast the weather from weather images
People memorize learned knowledge in the brain, how to memorize learned knowledge in machine learning
Paper: Music processing in the brain
How to make a face image data set used in machine learning (3: Face image generation from candidate images Part 1)
Image recognition model using deep learning in 2016
Image alignment: from SIFT to deep learning
"Deep Learning from scratch" in Haskell (unfinished)
Simple code that gives a score of 0.81339 in Kaggle's Titanic: Machine Learning from Disaster
How to collect machine learning data
How to send a visualization image of data created in Python to Typetalk
How to make a QGIS plugin (package generation)
[For recording] Keras image system Part 1: How to create your own data set?
How to set up a Google Colab environment with Coursera's advanced machine learning courses
How to split machine learning training data into objective variables and others in Pandas
How to scrape image data from flickr with python
How to make a shooting game with toio (Part 1)
Machine learning beginners try to make a decision tree
How to take a captured image from a video (OpenCV)
An introduction to machine learning from a simple perceptron
How to build Anaconda virtual environment used in Azure Machine Learning and link with Jupyter
Try running wav2pix to generate a face image from voice (animation face generation is also available)
How to interactively draw a machine learning pipeline with scikit-learn and save it in HTML
How to get only the data you need from a structured data set using a versatile method
How to plot the distribution of bacterial composition from Qiime2 analysis data in a box plot
[Part 1] Use Deep Learning to forecast the weather from weather images
[Part 3] Use Deep Learning to forecast the weather from weather images
How to slice a block multiple array from a multiple array in Python
How to make a hacking lab-Kali Linux (2020.1) VirtualBox 64-bit Part 2-
How to increase the number of machine learning dataset images
[Part 2] Use Deep Learning to forecast the weather from weather images
Data set for machine learning
Used in machine learning EDA
Machine learning experience (first part) in just a few lines. Explain PyCaret in detail. From dataset preparation to accuracy comparison of multiple models.
[Tutorial] Make a named entity extractor in 30 minutes using machine learning
How to make a string into an array or an array into a string in Python
How about Anaconda for building a machine learning environment in Python?
How to get a string from a command line argument in python
<Pandas> How to handle time series data in a pivot table
How to make a .dylib library from a .a library on OSX (El Capitan)
How to make a unit test Part.1 Design pattern for introduction
How to create a large amount of test data in MySQL? ??
People memorize learned knowledge in the brain, how to memorize learned knowledge in machine learning
How to create a serverless machine learning API with AWS Lambda
How to make a Japanese-English translation
Make a Santa classifier from a Santa image
How to collect images in Python
How to make a slack bot
How to make a crawler --Advanced
How to make a recursive function
How to make a deadman's switch
Machine learning in Delemas (data acquisition)
[Blender] How to make a Blender plugin
Preprocessing in machine learning 2 Data acquisition
How to make a crawler --Basic
Preprocessing in machine learning 4 Data conversion
Coursera Machine Learning Challenges in Python: ex6 (How to Adjust SVM Parameters)
Tweet in Chama Slack Bot ~ How to make a Slack Bot using AWS Lambda ~
In Vim: set to output html from markdown using pandoc with make
How to make a model for object detection using YOLO in 3 hours
Aiming to become a machine learning engineer from sales positions using MOOCs
How to get a value from a parameter store in lambda (using python)
How to make a container name a subdomain and make it accessible in Docker
I tried to make Kana's handwriting recognition Part 2/3 Data creation and learning
How to get a namespaced view name from a URL (path_info) in Django
[Development environment] How to create a data set close to the production DB