I tried face recognition from the video (OpenCV: python version)

Trigger

How to save only a part of a long video using OpenCV How to take a captured image from a video (OpenCV) I used OpenCV from the candidacy video to prepare the environment that seems to be necessary for image processing. From here, I will try to process using functions and libraries. The first is face recognition.

Preparation

Since it is developed on Mac, install it from Homebrew.

brew install opencv

Therefore, under / usr / local / Cellar / opencv / 4.1.1_2 / share / opencv4 / haarcascades /, there is a part called a cascade (distributor) that is necessary for recognizing the face and the whole body. The types are described in [Face recognition using OpenCV (Haar-like feature classifier)](https://qiita.com/hitomatagi/items/04b1b26c1bc2e8081427#haar-like feature classifier).

here,

--haarcascade_frontalface_alt_tree.xml front face --haarcascade_eye.xml Both eyes --haarcascade_righteye_2splits.xml right eye --haarcascade_lefteye_2splits.xml left eye

I will try using.

Process flow

here

  1. The face is in the frame
  2. Cut out an image that will be the ROI (Region of Interest) from 1. Divided into right area and left area respectively (Baron Ashura method).
  3. Determine if both the right and left eyes are visible in the image in 2.
  4. If the conditions of 3 are met, enclose the face, right eye and left eye with a rectangle.

Is being processed.

development of

Developed based on the above processing flow.

Version judged by dividing the right eye and the left eye


import cv2
import os
import time

if __name__ == '__main__':

	cap = cv2.VideoCapture('one_minutes.mp4')

	cap_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
	cap_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
	fps = cap.get(cv2.CAP_PROP_FPS)

	fourcc = cv2.VideoWriter_fourcc('m','p','4','v')
	writer = cv2.VideoWriter('detect_face.mp4',fourcc, fps, (cap_width, cap_height))

	cascade_base_path = "/usr/local/Cellar/opencv/4.1.1_2/share/opencv4/haarcascades/"
    #Get ready cascade
	face_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_frontalface_alt_tree.xml'))
	right_eye_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_righteye_2splits.xml'))
	left_eye_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_lefteye_2splits.xml'))

	start = time.time()

	try :
		while True:

			if not cap.isOpened():
				break

			ret, frame = cap.read()

			img_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
            # 1.The face is in the frame
			face_points = face_cascade.detectMultiScale(img_gray)
			
			for (fx,fy,fw,fh) in face_points:
				
                #2. ROI(Region of Interest:Target area)Cut out the image that becomes
                #Divided into right area and left area respectively(Baron Ashura method)
				width_center = fx + int(fw * 0.5)
				face_right_gray = img_gray[fy:fy+fh, fx:width_center]
				face_left_gray = img_gray[fy:fy+fh, width_center:fx+fw]
				#3.Determine if both right and left eyes are visible
				right_eye_points = right_eye_cascade.detectMultiScale(face_right_gray)
				left_eye_points = left_eye_cascade.detectMultiScale(face_left_gray)

				if 0 < len(right_eye_points) and 0 < len(left_eye_points):
					(rx,ry,rw,rh) = right_eye_points[0]
					(lx,ly,lw,lh) = left_eye_points[0]
				        
                    #4.Surround the face, right eye and left eye with a rectangle
					#Right eye is orange
					cv2.rectangle(frame,(fx+rx,fy+ry),(fx+rx+rw,fy+ry+rh),(0,255,255),2)
					#Left eye is red
					cv2.rectangle(frame,(width_center+lx,fy+ly),(width_center+lx+lw,fy+ly+lh),(0,0,255),2)
					#The whole face is green
					cv2.rectangle(frame,(fx,fy),(fx+fw,fy+fh),(0,255,0),2)

			writer.write(frame)
	except cv2.error as e:
		print(e)

	print("processing time{}Seconds".format(time.time() - start))
	writer.release()
	cap.release()

For comparison, I will also list the one using both eyes haarcascade_eye.xml.

Both eyes version


import cv2
import os
import time

if __name__ == '__main__':

	cap = cv2.VideoCapture('one_minutes.mp4')

	cap_width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
	cap_height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
	fps = cap.get(cv2.CAP_PROP_FPS)

	fourcc = cv2.VideoWriter_fourcc('m','p','4','v')
	writer = cv2.VideoWriter('detect_face_2.mp4',fourcc, fps, (cap_width, cap_height))

	cascade_base_path = "/usr/local/Cellar/opencv/4.1.1_2/share/opencv4/haarcascades/"

	face_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_frontalface_alt_tree.xml'))
	eyes_cascade = cv2.CascadeClassifier(os.path.join(cascade_base_path, 'haarcascade_eye.xml'))

	start = time.time()

	try :
		while True:

			if not cap.isOpened():
				break

			ret, frame = cap.read()

			img_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
			face_points = face_cascade.detectMultiScale(img_gray)
			
			for (fx,fy,fw,fh) in face_points:
				
				face_gray = img_gray[fy:fy+fh, fx:fx+fw]				
				eyes_points = eyes_cascade.detectMultiScale(face_gray)
				
				if 0 < len(eyes_points):
					for (ex,ey,ew,eh) in eyes_points:
						#Eyes are orange
						cv2.rectangle(frame,(fx+ex,fy+ey),(fx+ex+ew,fy+ey+eh),(0,255,255),2)
						#The whole face is green
						cv2.rectangle(frame,(fx,fy),(fx+fw,fy+fh),(0,255,0),2)

			writer.write(frame)
	except cv2.error as e:
		print(e)

	print("processing time{}Seconds".format(time.time() - start))
	writer.release()
	cap.release()

result

face_main.gif

face_sub.gif

What I found

――The accuracy was higher when the right eye and the left eye were judged separately. Especially when I saw the competition, I recognized it more than both eyes! -Although it is recognition of both eyes, it will be misrecognized other than the eyes ――Even if your face is tilted a little, the accuracy will drop sharply. I haven't broken N ○ H: tired_face: It doesn't seem to recognize it without a cascade that trains tilted and resized images. It may be best to use Deep Learning now. Certainly, I think that the accuracy seems to be good because I am learning after duplicating one piece of data in the library by tilting, resizing, blurring, etc .: thinking: (I have to remember new things again. Is it ...: weary: I don't have the energy and physical strength: thermometer_face :) --The processing time took 276.574 seconds. (About 4 minutes) If it is C ++, it will end soon.

in conclusion

A few days ago, Try real-time face detection with OpenCV was posted and was dropped: scream: It took more time to put together than development: weary:

Referenced link

-[Face recognition using OpenCV (Haar-like feature classifier)](https://qiita.com/hitomatagi/items/04b1b26c1bc2e8081427#haar-like%E7%89%B9%E5%BE%B4%E5% 88% 86% E9% A1% 9E% E5% 99% A8) -[Explanation for beginners] openCV face detection mechanism and practice (detectMultiScale) --Detailed explanation. It seems that the accuracy is not good if the object you want to track is rotating

Recommended Posts

I tried face recognition from the video (OpenCV: python version)
I tried to display the video playback time (OpenCV: Python version)
I tried face recognition with OpenCV
I tried to make the political broadcast video like IPPON Grand Prix (OpenCV: Python version)
I tried "smoothing" the image with Python + OpenCV
I tried "differentiating" the image with Python + OpenCV
I tried "binarizing" the image with Python + OpenCV
I tried face recognition using Face ++
I tried "gamma correction" of the image with Python + OpenCV
I tried using the Python library from Ruby with PyCall
I tried face recognition of the laughter problem using Keras.
Try face recognition with python + OpenCV
Face recognition with camera with opencv3 + python2.7
I tried changing the python script from 2.7.11 to 3.6.0 on windows10
I tried to cut out a still image from the video
I tried non-photorealistic rendering with Python + opencv
I tried using UnityCloudBuild API from Python
[python, openCV] base64 Face recognition with images
Python: I tried the traveling salesman problem
I tried the Python Tornado Testing Framework
[Python] I tried substituting the function name for the function name
I tried hitting the Qiita API from go
vprof --I tried using the profiler for Python
I tried object detection using Python and OpenCV
I tried simulating the "birthday paradox" in Python
I tried the least squares method in Python
I tried Python! ] I graduated today from "What is Python! Python!"!
I tried python programming for the first time.
I tried debugging from Python via System Console
Try the free version of Progate [Python I]
I tried using the Datetime module by Python
I tried using the image filter of OpenCV
[IBM Cloud] I tried to access the Db2 on Cloud table from Cloud Funtions (python)
I tried running two Jetson Nano hardware PWMs from the Jetson.GPIO Python library.
[Python] I tried to get the type name as a string from the type function
[Python] I tried the same calculation as LSTM predict with from scratch [Keras]
I tried Python> autopep8
I tried Python> decorator
I tried to graph the packages installed in Python
I tried Python on Mac for the first time.
I tried pipenv and asdf for Python version control
I tried to detect the iris from the camera image
[Python] Extract the video ID from the YouTube video URL [Note]
I tried to touch the CSV file with Python
[OpenCV / Python] I tried image analysis of cells with OpenCV
I tried running python etc. from a bat file
I tried to solve the soma cube with python
I tried python on heroku for the first time
I wanted to use the Python library from MATLAB
[Python] I tried to graph the top 10 eyeshadow rankings
I tried to solve the problem with Python Vol.1
Python OpenCV tried to display the image in text.
I tried "morphology conversion" of images with Python + OpenCV
I tried hitting the API with echonest's python client
I tried to summarize the string operations of Python
I raised the Python version from 2 to 3, but every time I restart the ubuntu terminal, the version remains 2.
(Python: OpenCV) I tried to output a value indicating the distance between regions while binarizing the video in real time.
I tried the site "Deploy Azure virtual machine from Go"
I tried to find the entropy of the image with python
I tried fp-growth with python
I tried scraping with Python