Don't you want to say that you made a face recognition program?
Face recognition in the times
Shenzhen is one of the most IT-developed cities in China. According to what I hear, you can make payments by simply holding your face over from convenience store payments to vending machines. Whether it's good or bad, it's amazing. It's futuristic and cool. Then make me too! I made it last year with a glue like w, but I couldn't make it into an article, so I decided to make it this time. The systems and programs introduced this time are the first works that touched Python properly, and the first memorable works that touched machine learning.
Overview
After taking an image of your face using the in-camera of your PC, processing and padding the image using the library OpenCV, you can learn the face with traincascade and create a program to identify the individual. The language used was Python3.
Program flow
This time, we created two types of face image processing, learning program and face recognition program.
Face image processing, learning program flow
① Start the PC camera with video capture (2) Save 40 face-recognized frames as images. ③ Add 40 images in various forms to increase the learning data. ④ Generate learning commands ⑤ Type the generated command into the console to start learning. ⑥ Generate a cascade file after learning is completed
Face recognition program flow
① Read the registrant's cascade file (2) Pass each frame through a cascade file to identify individuals.
Folder structure
Face image processing, learning program
face_learner/
├─face_learner.py
├─haarcascade_frontalface_default.xml
├─neg.txt
├─pos.txt
├─cascade/
├─neg/
├─neg1.jpg
├─neg2.jpg
├─neg3.jpg
…
└─neg265.jpg
├─pic/
├─pic1.jpg
├─pic2.jpg
├─pic3.jpg
…
└─pic1639.jpg
└─pos.vec
This folder structure is only the files around the learning program. The personal identification program is not included.
Program description (learning program)
Please refer to the commented out explanation of what to do easily in the source code. ..
Two types of correct image (positive image) and incorrect image (negative image) are required for learning. The correct image is the face image you want to learn, and you also need the coordinate information of where the face is in the image. The incorrect image can be anything other than the correct image. This time, 265 cute animal images are prepared and stored in the neg directory in advance.
First of all, we will start by collecting these correct images. For the correct answer image, first take 40 face images and then increase the number to about 1640 to improve the learning accuracy. The more the number of correct images, the higher the learning accuracy, so it is advisable to increase the number as appropriate. I feel that there is a consultation with the PC specifications. Since you will be learning a new face, you need to delete all the unnecessary data you used before. You should be able to handle it smartly here too (laughs). Collecting facial images is the main purpose of this program. The face is recognized using the pre-loaded cascading file for face recognition, and 40 face images are first taken. Start adding water from here. Water is added by adding various additions to the image. It seems that there is already a tool in this part, so it is absolutely quick to use it. This time, I also made my own programming practice.
[Type of image modification] • Flip horizontal • Saturation change • Contrast change • Change brightness • Sharpness change • ± 15 degree rotation
There is a lot of room for improvement in the processing here. First of all, it is absolutely good to make each processing part functional. In this program, the number of photos is counted and added each time, so it is better to apply it a little smarter. After collecting the images, it is necessary to create a text file that summarizes the information of the correct image. This is the path of the image file, which part of which image has the face, and the coordinate information of the four quadrilaterals that surround the part with the face. The format is [Number of face faces Coordinates (x) Coordinates (y) Coordinates (w) Coordinates (h)]. w is the width and h is the height. An example is shown below.
pos.txt
pic/pic1389.jpg 1 454 328 547 547
pic/pic302.jpg 1 565 85 380 380
pic/pic464.jpg 1 947 389 31 31
pic/pic470.jpg 1 663 238 341 341
pic/pic1438.jpg 1 658 341 540 540
pic/pic316.jpg 1 620 88 376 376
pic/pic1376.jpg 1 795 414 376 376
pic/pic1410.jpg 1 476 408 379 379
pic/pic1404.jpg 1 497 394 383 383
pic/pic1362.jpg 1 754 400 372 372
pic/pic869.jpg 1 625 235 420 420
pic/pic855.jpg 1 380 185 381 381
pic/pic666.jpg 1 361 241 424 424
pic/pic100.jpg 1 312 91 360 360
pic/pic64.jpg 1 622 101 371 371
pic/pic114.jpg 1 279 84 375 375
pic/pic70.jpg 1 618 89 382 382
pic/pic672.jpg 1 1213 216 43 43
pic/pic672.jpg 1 369 274 385 385
pic/pic882.jpg 1 721 313 353 353
.
.
.
Once you've generated the text, all you have to do is generate the command. First, a command to convert to a vector file using the information of the correct image. The next command to learn. Python has the ability to execute Linux commands from a python program, such as the os.system () and subprocess.call () functions, but I didn't get it so I can do it manually. I am trying to generate a command to. If anyone can tell me about this part, I will study! When the learning is finished, the cascade.xml file is created in the cascade directory, so read it and perform personal identification. It will be easier to understand if the file name is an individual name as appropriate.
Program description (discrimination program)
Here, individual identification is performed using the cascade file created earlier. The only library loaded by this program is OpenCV (cv2). Pass the frame captured by the camera through the cascade file each time, and when the face is raised, enclose the closed part with a frame. First, when a person's face is recognized, it is surrounded by a white frame, and when a person is recognized, it is surrounded by a blue frame, and the name is drawn on it.
With this kind of feeling, you can identify individuals in real time with an in-camera!
Actual program
I'm really reluctant to expose dirty code, but I'm going to publish the whole code, believing it will help. Please note that it is long enough to die.
Image collection + learning program
face_leaner.py
import os
import re
import numpy as np
import time
import glob
import shutil
import PIL.Image
from PIL import ImageEnhance
import subprocess
import cv2
#uer_name = input("Who are you")
cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
#----------------------------------------------
#mov, pic,Clear pos
os.chdir('cascade')
for x in glob.glob('*.xml'):
os.remove(x)
os.chdir('../')
os.chdir('pic')
for x in glob.glob('*.jpg'):
os.remove(x)
os.chdir('../')
os.chdir('pos')
for x in glob.glob('*.jpg'):
os.remove(x)
os.chdir('../')
os.remove('pos.txt')
#----------------------------------------------
#pos.Create txt
f = open('pos.txt', 'a')
#----------------------------------------------
#Collect facial images
cap = cv2.VideoCapture(0)
cascade_path1 = "haarcascade_frontalface_default.xml"
cascade_path2 = 'lbpcascade_profileface.xml'
cascade1 = cv2.CascadeClassifier(cascade_path1)
cascade2 = cv2.CascadeClassifier(cascade_path2)
color = (255,255,255)
picture_num = 1
while True:
ret, frame = cap.read()
facerect1 = cascade1.detectMultiScale(frame, scaleFactor=1.7, minNeighbors=4, minSize=(100,100))
font = cv2.FONT_HERSHEY_SIMPLEX
cv2.putText(frame, str(picture_num), (10,500), font, 4,(0,0,0),2,cv2.LINE_AA)
if len(facerect1) > 0:
for (x,y,w,h) in facerect1:
#cv2.rectangle(frame,(x,y),(x+w,y+h),(255,0,0),2)
picture_name = 'pic/pic' + str(picture_num) + '.jpg'
cv2.imwrite(picture_name, frame)
#text = picture_name + ' 1 ' + str(x) + ' ' + str(y) + ' ' + str(w) + ' ' + str(h) + '\n'
#f.write(text)
picture_num = picture_num + 1
cv2.imshow("frame", frame)
if picture_num == 41:
break
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
#----------------------------------------------
#Start padding
#Count the number of pictures in pictures
dir = os.getcwd()
dirPic = dir + "/pic"
files = os.listdir(dirPic)
count = 0
for file in files:
count = count + 1
os.chdir('pic')
#Number of photos
imageNum = count
#Flip horizontal
for num in range(1, count+1):
name = 'pic' + str(num) + '.jpg'
if os.path.exists(name) :
pass
else :
continue
if os.path.getsize(name) == 0:
os.remove(name)
continue
img = cv2.imread(name)
yAxis = cv2.flip(img, 1)
newName = 'pic' + str(imageNum) + '.jpg'
cv2.imwrite(newName,yAxis)
imageNum = imageNum + 1
print('OK')
print('NEXT STAGE')
#Count the number of photos
dir = os.getcwd()
files = os.listdir(dir)
count = 0
for file in files:
count = count + 1
print(count)
#Number of photos
imageNum = count
#Fix imageNum
picNum = imageNum
SATURATION = 0.5
CONTRAST = 0.5
BRIGHTNESS = 0.5
SHARPNESS = 2.0
#Change the saturation
for num in range(1, count+1):
name = 'pic' + str(num) + '.jpg'
if os.path.exists(name) :
pass
else :
print('NO')
continue
if os.path.getsize(name) == 0:
os.remove(name)
continue
img = PIL.Image.open(name)
saturation_converter = ImageEnhance.Color(img)
saturation_img = saturation_converter.enhance(SATURATION)
newName = 'pic' + str(imageNum) + '.jpg'
saturation_img.save(newName)
imageNum = imageNum + 1
print('OK')
print('NEXT STAGE')
#Count the number of photos
dir = os.getcwd()
files = os.listdir(dir)
count = 0
for file in files:
count = count + 1
print(count)
#Number of photos
imageNum = count
count = picNum
#Change the contrast
for num in range(1, count+1):
name = 'pic' + str(num) + '.jpg'
if os.path.exists(name) :
pass
else :
print('NO')
continue
if os.path.getsize(name) == 0:
os.remove(name)
continue
img = PIL.Image.open(name)
contrast_converter = ImageEnhance.Contrast(img)
contrast_img = contrast_converter.enhance(CONTRAST)
newName = 'pic' + str(imageNum) + '.jpg'
contrast_img.save(newName)
imageNum = imageNum + 1
print('OK')
print('NEXT STAGE')
#Count the number of photos
dir = os.getcwd()
files = os.listdir(dir)
count = 0
for file in files:
count = count + 1
print(count)
#Number of photos
imageNum = count
count = picNum
#Change the brightness
for num in range(1, count+1):
name = 'pic' + str(num) + '.jpg'
if os.path.exists(name) :
pass
else :
print('NO')
continue
if os.path.getsize(name) == 0:
os.remove(name)
continue
img = PIL.Image.open(name)
brightness_converter = ImageEnhance.Brightness(img)
brightness_img = brightness_converter.enhance(BRIGHTNESS)
newName = 'pic' + str(imageNum) + '.jpg'
brightness_img.save(newName)
imageNum = imageNum + 1
print('OK')
print('NEXT STAGE')
#Count the number of photos
dir = os.getcwd()
files = os.listdir(dir)
count = 0
for file in files:
count = count + 1
print(count)
#Number of photos
imageNum = count
count = picNum
#Change sharpness
for num in range(1, count+1):
name = 'pic' + str(num) + '.jpg'
if os.path.exists(name) :
pass
else :
print('NO')
continue
if os.path.getsize(name) == 0:
os.remove(name)
continue
img = PIL.Image.open(name)
sharpness_converter = ImageEnhance.Sharpness(img)
sharpness_img = sharpness_converter.enhance(SHARPNESS)
newName = 'pic' + str(imageNum) + '.jpg'
sharpness_img.save(newName)
imageNum = imageNum + 1
print('OK')
print('NEXT STAGE')
#Count the number of photos
dir = os.getcwd()
files = os.listdir(dir)
count = 0
for file in files:
count = count + 1
print(count)
#Number of photos
imageNum = count
#Fix imageNum
picNum = imageNum
#15 degree rotation
for num in range(1, count+1):
name = 'pic' + str(num) + '.jpg'
if os.path.exists(name) :
pass
else :
print('NO')
continue
if os.path.getsize(name) == 0:
os.remove(name)
continue
#Image loading
img = cv2.imread(name)
h, w = img.shape[:2]
size = (w, h)
#Specifying the rotation angle
angle = 15
angle_rad = angle/180.0*np.pi
#Calculate image size after rotation
w_rot = int(np.round(h*np.absolute(np.sin(angle_rad))+w*np.absolute(np.cos(angle_rad))))
h_rot = int(np.round(h*np.absolute(np.cos(angle_rad))+w*np.absolute(np.sin(angle_rad))))
size_rot = (w_rot, h_rot)
#Rotate around the center of the original image
center = (w/2, h/2)
scale = 1.0
rotation_matrix = cv2.getRotationMatrix2D(center, angle, scale)
#Add translation(rotation + translation)
affine_matrix = rotation_matrix.copy()
affine_matrix[0][2] = affine_matrix[0][2] -w/2 + w_rot/2
affine_matrix[1][2] = affine_matrix[1][2] -h/2 + h_rot/2
img_rot = cv2.warpAffine(img, affine_matrix, size_rot, flags=cv2.INTER_CUBIC)
cv2.imwrite(newName, img_rot)
newName = 'pic' + str(imageNum) + '.jpg'
saturation_img.save(newName)
imageNum = imageNum + 1
print('OK')
print('NEXT STAGE')
#Count the number of photos
dir = os.getcwd()
files = os.listdir(dir)
count = 0
for file in files:
count = count + 1
print(count)
#Number of photos
imageNum = count
#-15 degree rotation
for num in range(1, count+1):
name = 'pic' + str(num) + '.jpg'
if os.path.exists(name) :
pass
else :
print('NO')
continue
if os.path.getsize(name) == 0:
os.remove(name)
continue
#Image loading
img = cv2.imread(name)
h, w = img.shape[:2]
size = (w, h)
#Specifying the rotation angle
angle = -15
angle_rad = angle/180.0*np.pi
#Calculate image size after rotation
w_rot = int(np.round(h*np.absolute(np.sin(angle_rad))+w*np.absolute(np.cos(angle_rad))))
h_rot = int(np.round(h*np.absolute(np.cos(angle_rad))+w*np.absolute(np.sin(angle_rad))))
size_rot = (w_rot, h_rot)
#Rotate around the center of the original image
center = (w/2, h/2)
scale = 1.0
rotation_matrix = cv2.getRotationMatrix2D(center, angle, scale)
#Add translation(rotation + translation)
affine_matrix = rotation_matrix.copy()
affine_matrix[0][2] = affine_matrix[0][2] -w/2 + w_rot/2
affine_matrix[1][2] = affine_matrix[1][2] -h/2 + h_rot/2
img_rot = cv2.warpAffine(img, affine_matrix, size_rot, flags=cv2.INTER_CUBIC)
cv2.imwrite(newName, img_rot)
newName = 'pic' + str(imageNum) + '.jpg'
saturation_img.save(newName)
imageNum = imageNum + 1
print('OK')
print('NEXT STAGE')
#Count the number of photos
dir = os.getcwd()
files = os.listdir(dir)
count = 0
for file in files:
count = count + 1
print(count)
print('OK')
print('COMPLETE')
#------------------------------------------------------
#Text file creation
#cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
#os.chdir('pic')
for num in glob.glob('*.jpg'):
img = cv2.imread(num)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = cascade.detectMultiScale(gray)
for (x,y,w,h) in faces:
text = 'pic/' + num + ' 1 ' + str(x) + ' ' + str(y) + ' ' + str(w) + ' ' + str(h) + '\n'
f.write(text)
cmd = 'opencv_createsamples -info pos.txt -vec pos.vec -num ' + str(count)
print(cmd)
cmd = 'opencv_traincascade -data ./cascade -vec pos.vec -bg neg.txt -numPos 1500 numNeg 255'
print(cmd)
print('COMPLETE.')
Personal identification program
face_id.py
# coding: utf-8
import cv2
font = cv2.FONT_HERSHEY_SIMPLEX
if __name__ == "__main__":
cap = cv2.VideoCapture(0)
cascade_path_human = 'haarcascade_frontalface_default.xml'
cascade_path_hirosugu = "cascade_hirosugu.xml"
cascade_path_kenta = 'cascade_kenta.xml'
cascade_hirosugu = cv2.CascadeClassifier(cascade_path_hirosugu)
cascade_human = cv2.CascadeClassifier(cascade_path_human)
cascade_kenta = cv2.CascadeClassifier(cascade_path_kenta)
color = (255,0,0)
while True:
ret, frame = cap.read()
facerect_human = cascade_human.detectMultiScale(frame, scaleFactor=1.7, minNeighbors=4, minSize=(100,100))
facerect_hirosugu = cascade_hirosugu.detectMultiScale(frame, scaleFactor=1.7, minNeighbors=4, minSize=(100,100))
facerect_kenta = cascade_kenta.detectMultiScale(frame, scaleFactor=1.7, minNeighbors=4, minSize=(100,100))
if len(facerect_human) > 0:
for rect in facerect_human:
cv2.rectangle(frame, tuple(rect[0:2]), tuple(rect[0:2]+rect[2:4]), (255,255,255), thickness=2)
if len(facerect_hirosugu) > 0:
for rect in facerect_hirosugu:
cv2.rectangle(frame, tuple(rect[0:2]), tuple(rect[0:2]+rect[2:4]), color, thickness=2)
cv2.putText(frame, 'Hirosugu Takeshita', tuple(rect[0:2]), font, 2,(0,0,0),2,cv2.LINE_AA)
if len(facerect_kenta) > 0:
for rect in facerect_kenta:
cv2.rectangle(frame, tuple(rect[0:2]), tuple(rect[0:2]+rect[2:4]), color, thickness=2)
cv2.putText(frame, 'Kenta Suzuki', tuple(rect[0:2]), font, 2,(0,0,0),2,cv2.LINE_AA)
cv2.imshow("frame", frame)
#Press the q key to end the loop
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
By the way, it is recommended to put the generated cascade file (.xml learning file) in the same hierarchy as the face recognition program!
Finally
Right? Isn't it easy? This is a garbage program for professionals! You might think that, but you can call yourself an Ese ML engineer. If it is true, the accuracy will be further improved if you do the feature extraction and write the NN yourself (I don't know because I'm not so familiar with it).
Also, there are many articles about OpenCV, so please refer to that as well. Since this article is only a fairly thin trace, it is recommended that you thoroughly investigate the detailed mechanism and other commands during learning.
Well, this time it's a personal identification by machine learning, so I asked them to bring back only the knowledge that it would be easier to collect face images in this way, and I will make a more accurate face ID and introduce it. I would be grateful if you could!
Thank you for reading for a long time!
Recommended Posts