How to make a face image data set used in machine learning (2: Frame analysis of video to obtain candidate images)

Last time introduced how to collect candidate images using the Bing Image Search API. This time, I will introduce how to analyze and collect videos by frame analysis.

Development environment used this time

Benefits of extracting candidate images from videos

The video is basically the same mechanism as a flip book, and the movement is expressed by switching the still image within a short time. At this time, the still image that composes the video is the frame image, and the number of frame images per unit time that composes the video is It is called the frame rate and is expressed in units of fps (how many frame images are used per second).

In other words, if you extract the frame image of the scene in which the face is reflected from the video, you can secure some candidate images even from the video with the short scene. In addition, many videos such as DVDs are taken with the subject's face looking at the camera, and many of them have appropriate corrections such as the amount of light. One of the merits is that there is a high possibility that a face image suitable for learning can be extracted from the candidate images extracted from the video.

This time, I would like to extract a frame image that is a candidate image from the video using OpenCV that I used last time.

Get captured video with QuickTime Player

This time, I will capture the video using the free tool QuickTime Player.

You may find it inconvenient to capture the video, but the .mov format video captured and output by QuickTime Player has the advantage that the file can be read smoothly with OpenCV. (If you already have a video that can be read by OpenCV, skip this step.)

Also, when extracting all frame images from a long-time video such as a DVD, the number is enormous and the face may not be properly reflected in all the frame images. Therefore, by partially capturing the video scene in which the face is reflected from the video, the aim is to efficiently extract the frame image in which the face is reflected. Furthermore, if this method is used, candidate images can be extracted from moving images published on the web.

※※※ Caution ※※※ ** The video captured this time is intended for use in machine learning. ** ** ** Please refrain from redistributing the captured video. ** ** ** Also, when capturing videos posted on the web, ** ** Please do not touch the rules of each video site that publishes the video! ** **

As for the capture method, detailed explanation is posted in "Recording the screen" of Official site. The explanation is omitted here. If you need to download and install QuickTime Player, please go to here. There is no audio in the captured video, but this time there is no problem because no audio is required.

Acquire frame image as a candidate image from captured video with OpenCV

After generating the captured video, extract the frame image that is a candidate image from it. The following is a code example that extracts a frame image and saves it as a candidate image.

# -*- coding: utf-8 -*-

import cv2


def movie_to_image(num_cut):

    video_path = '/hoge/hoge.mov'   #Captured video path (including file name)
    output_path = '/hoge/save/'  #Folder path to output
    
    #Captured video read (capture structure generation)
    capture = cv2.VideoCapture(video_path)

    img_count = 0  #Number of saved candidate images
    frame_count = 0  #Number of frame images read

	#Loop as long as there is a frame image
    while(capture.isOpened()):
    	 #Get one frame image
        ret, frame = capture.read()
        if ret == False:
            break

        #Saves the specified number of frame images by thinning them out
        if frame_count % num_cut == 0:
            img_file_name = output_path + str(img_count) + ".jpg "
            cv2.imwrite(img_file_name, frame)
            img_count += 1
            
        frame_count += 1

    #Capture structure open
    capture.release()


if __name__ == '__main__':    
	
	#Frame image extraction with the thinning number set to 10
	movie_to_image(int(10))
			

The argument given to the movie_to_image method is the number to thin out the frame image. As explained above, in the case of a long-time video, saving all the frame images as candidate images will result in a huge number of images. Also, if the motion in the video is extremely slow, multiple face frame images with almost the same composition will be generated. In some cases, you may dislike it. By thinning out the frame images to be saved there to some extent, those problems are solved.

Therefore, in the case of a video with a frame rate of 29.97fps, If you set the thinning number to 30, you can get about one frame image at 1 second intervals in the video.

Real-time candidate image acquisition using a webcam

In the above code, a candidate image was generated from the captured video file, but with a slight change in this code You can also generate candidate images in real time while shooting a video with a webcam.

For example, to use the in-camera that comes standard with the MacBook Air, All you have to do is change the above code as follows.

	#capture = cv2.VideoCapture(video_path)
	
	#Give the camera device number
	capture = cv2.VideoCapture(0)

You can also use a USB-connected webcam, but depending on the camera, the device may not be recognized and you may not be able to use it.

Also, if there is no machine spec to some extent, real-time processing while shooting may not be able to catch up. The problem tends to be more pronounced, especially when saving the entire number of frames. In that case, please solve it by adjusting the thinning number.     The above is simple, but it was an explanation of an example of collecting candidate images by frame analysis of a video.

Recommended Posts

How to make a face image data set used in machine learning (2: Frame analysis of video to obtain candidate images)
How to send a visualization image of data created in Python to Typetalk
How to increase the number of machine learning dataset images
How to plot the distribution of bacterial composition from Qiime2 analysis data in a box plot
How to create a large amount of test data in MySQL? ??
How to collect machine learning data
[Part 1] Use Deep Learning to forecast the weather from weather images
[Part 3] Use Deep Learning to forecast the weather from weather images
[Part 2] Use Deep Learning to forecast the weather from weather images
People memorize learned knowledge in the brain, how to memorize learned knowledge in machine learning
Paper: Music processing in the brain
How to make a face image data set used in machine learning (3: Face image generation from candidate images Part 1)
Image recognition model using deep learning in 2016
Image alignment: from SIFT to deep learning
"Deep Learning from scratch" in Haskell (unfinished)
Simple code that gives a score of 0.81339 in Kaggle's Titanic: Machine Learning from Disaster
Memorandum of means when you want to make machine learning with 50 images
Preprocessing in machine learning 1 Data analysis process
Basic data frame operations written by beginners in a week of learning Python
Full disclosure of methods used in machine learning
Summary of evaluation functions used in machine learning
A well-prepared record of data analysis in Python
A story about data analysis by machine learning
How to set up a Google Colab environment with Coursera's advanced machine learning courses
How to split machine learning training data into objective variables and others in Pandas
Create a dataset of images to use for learning
List of Python code used in big data analysis
Machine learning beginners try to make a decision tree
Basics of PyTorch (2) -How to make a neural network-
How to take a captured image from a video (OpenCV)
How to build Anaconda virtual environment used in Azure Machine Learning and link with Jupyter
How to interactively draw a machine learning pipeline with scikit-learn and save it in HTML
Examples and countermeasures for "A value is trying to be set on a copy of a slice from a Data Frame." Warning in pandas
[Python] How to make a list of character strings character by character
How to develop in a virtual environment of Python [Memo]
How to get a list of built-in exceptions in python
Summary of how to write .proto files used in gRPC
How to get a quadratic array of squares in a spiral!
Inflated learning image
Creating learning data for face image dataset sorting (# 1)
Data set for machine learning
Deep learning image recognition 1 theory
Python data analysis learning notes
Image data type conversion [Python]