I tried face recognition of the laughter problem using Keras.

Identify the laughter problem with image recognition

I was free in Corona so I made it quickly. Most of the code is reused when the image of Keyakizaka46 is recognized. If you want to recognize your favorite images with deep learning, please come! So that you can do it even for the first time, the parts that are likely to trip are also listed! !! [I think this is all you need to do with python preferences! ] (https://prog-8.com/docs/python-env)

directory

/bakusyomondai
 /data
  /TANAKA
  /ota
 /face
  /ota
  /TANAKA
 /train
  /ota
  /TANAKA
 /test
  /ota
  /TANAKA
 
 get_image.py
 detect_face/py
 devide_test_train.py
 inflation.py
 learn2.py
 Bakunin.py

For example, in the case of mac, it is as follows. Screenshot 2020-04-19 16.16.27.png

スクリーンショット 2020-04-19 16.14.44.png

table of contents

1, download images to learn 2, Face detection of downloaded image 3, Inflating the image to be learned 4, Learning (Deep Learninng production) 5, Discriminate images using the learned model

1, download images to learn

First, create and save the following file called get_image.py. Click here for how to download automatically from google search images

get_image.py


from icrawler.builtin import BingImageCrawler
import sys
import os

argv = sys.argv

if not os.path.isdir(argv[1]):
    os.makedirs(argv[1])


crawler = BingImageCrawler(storage = {"root_dir" : argv[1]})
crawler.crawl(keyword = argv[2], max_num = 1000)

If you execute this python file in the terminal as follows, the image will be downloaded. For example, if you want to download Tanaka from Bakusho Mondai,

$ python */get_image.py */data/TANAKA Bakusho Mondai Tanaka

The download will start when you execute.

As an image

$python */get_image.py Directory of the location you want to save Name you want to search

By the way

I think that a lot of asterisks will appear in the code this time like * / get_image.py, but this time I dare to hide it with an asterisk. Please refer to the link below when checking the file directory on your own computer. For windows For Mac

2. Face detection of downloaded images

I think the contents of the file created earlier are as follows. スクリーンショット 2020-04-19 6.09.36.png If there is a photo other than Mr. Ota, I deleted it, and if there are people other than Mr. Ota, I cut out the face of only Mr. Ota.

First of all, this time, I used the following code.

detect_face.py


import glob
import os
import cv2

names = ['ota','TANAKA']
out_dir = "*/bakusyomondai/face/"
os.makedirs(out_dir, exist_ok=True)
for i in range(len(names)):
   
    in_dir = "*/bakusyomondai/data/"+names[i]+"/*.jpg "
    in_jpg = glob.glob(in_dir)
    os.makedirs(out_dir + names[i], exist_ok=True)
    # print(in_jpg)
    print(len(in_jpg))
    for num in range(len(in_jpg)):
        image=cv2.imread(str(in_jpg[num]))
        if image is None:
            print("Not open:",num)
            continue

        image_gs = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        cascade = cv2.CascadeClassifier("*/haarcascade_frontalface_alt.xml")

       
        face_list=cascade.detectMultiScale(image_gs, scaleFactor=1.1, minNeighbors=2,minSize=(64,64))
       
        if len(face_list) > 0:
            for rect in face_list:
                x,y,width,height=rect
                image = image[rect[1]:rect[1]+rect[3],rect[0]:rect[0]+rect[2]]
                if image.shape[0]<64:
                    continue
                image = cv2.resize(image,(64,64))
                
                fileName=os.path.join(out_dir+"/"+names[i],str(num)+".jpg ")
                cv2.imwrite(str(fileName),image)
                print(str(num)+".I saved the jpg.")
       
        else:
            print("no face")
            continue
        print(image.shape)

When you run this code

スクリーンショット 2020-04-19 16.16.27.png In this way, an image in which only the face of the image downloaded earlier is cut out is created in the face file.

Since a cascade file called haarcascade_frontalface_alt.xml is used for face detection, Download from here!

3. Divide the image into training data and test data.

devide_test_train.py


import shutil
import random
import glob
import os
names = ['ota','TANAKA']
os.makedirs("*/bakusyomondai/test", exist_ok=True)

for name in names:
    in_dir = "*/bakusyomondai/face/"+name+"/*"
    in_jpg=glob.glob(in_dir)
    img_file_name_list=os.listdir("*/bakusyomondai/face/"+name+"/")
    #img_file_name_Shuffle list, 20% of which test_Put in image directory
    random.shuffle(in_jpg)
    os.makedirs('*/bakusyomondai/test/' + name, exist_ok=True)
    for t in range(len(in_jpg)//5):
        shutil.move(str(in_jpg[t]), "*/bakusyomondai/test/"+name)

4. Inflating training data

As expected, the number of images is small, so I will inflate the images.

inflation.py


import os
import cv2
import glob
from scipy import ndimage

names = ['ota','TANAKA']
os.makedirs("*/bakusyomondai/train", exist_ok=True)
for name in names:
    in_dir = "*/bakusyomondai/face/"+name+"/*"
    out_dir = "*/bakusyomondai/train/"+name
    os.makedirs(out_dir, exist_ok=True)
    in_jpg=glob.glob(in_dir)
    img_file_name_list=os.listdir("*/bakusyomondai/face/"+name+"/")
    for i in range(len(in_jpg)):
        #print(str(in_jpg[i]))
        img = cv2.imread(str(in_jpg[i]))
        
        for ang in [-10,0,10]:
            img_rot = ndimage.rotate(img,ang)
            img_rot = cv2.resize(img_rot,(64,64))
            fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+".jpg ")
            cv2.imwrite(str(fileName),img_rot)
           
            img_thr = cv2.threshold(img_rot, 100, 255, cv2.THRESH_TOZERO)[1]
            fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+"thr.jpg ")
            cv2.imwrite(str(fileName),img_thr)
            
            img_filter = cv2.GaussianBlur(img_rot, (5, 5), 0)
            fileName=os.path.join(out_dir,str(i)+"_"+str(ang)+"filter.jpg ")
            cv2.imwrite(str(fileName),img_filter)

When executed, the image will be inflated as shown below. スクリーンショット 2020-04-19 16.29.55.png

5, start of learning

learn2.py


import keras
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.convolutional import Conv2D, MaxPooling2D
from keras.layers.core import Dense, Dropout, Activation, Flatten
import numpy as np
from sklearn.model_selection import train_test_split
from PIL import Image
import glob

folder = ['ota','TANAKA']
image_size = 50

X_train = []
y_train = []
for index, name in enumerate(folder):
    dir = "*/bakusyomondai/train/" + name
    files = glob.glob(dir + "/*.jpg ")
    for i, file in enumerate(files):
        image = Image.open(file)
        image = image.convert("RGB")
        image = image.resize((image_size, image_size))
        data = np.asarray(image)
        X_train.append(data)
        y_train.append(index)

X_train = np.array(X_train)
y_train = np.array(y_train)

folder = ['ota','TANAKA']
image_size = 50

X_test = []
y_test = []
for index, name in enumerate(folder):
    dir = "*/bakusyomondai/test/" + name
    files = glob.glob(dir + "/*.jpg ")
    for i, file in enumerate(files):
        image = Image.open(file)
        image = image.convert("RGB")
        image = image.resize((image_size, image_size))
        data = np.asarray(image)
        X_test.append(data)
        y_test.append(index)

X_test = np.array(X_test)
y_test = np.array(y_test)

X_train = X_train.astype('float32')
X_train = X_train / 255.0

X_test = X_test.astype('float32')
X_test = X_test / 255.0

#Convert the correct label format
y_train = np_utils.to_categorical(y_train, 2)
#Convert the correct label format
y_test = np_utils.to_categorical(y_test, 2)

#Build CNN
model = Sequential()

model.add(Conv2D(32, (3, 3), padding='same',input_shape=X_train.shape[1:]))
model.add(Activation('relu'))
model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Conv2D(64, (3, 3), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(2))
model.add(Activation('softmax'))

#compile
model.compile(loss='categorical_crossentropy',optimizer='SGD',metrics=['accuracy'])

history = model.fit(X_train, y_train, epochs=100)
print(model.evaluate(X_test, y_test))
model.save("*/bakusyomonadai/bakusyo_model.h5")
スクリーンショット 2020-04-19 16.34.45.png It did it like this. For example, if you want to study with four people, "'Nobita',' Shizuka',' Suneo',' Gian'"
y_train = np_utils.to_categorical(y_train, 4)
#Convert the correct label format
y_test = np_utils.to_categorical(y_test, 4)

model.add(Dense(4))

If you change this part, it will work! !!

6. Discriminate images using the learned model

kakunin.py


from keras.models import load_model
import numpy as np
from keras.preprocessing.image import img_to_array, load_img

jpg_name = '*/hirate/000002.jpg'
my_model='*/bakusyomondai/bakusyo_model.h5'

model=load_model(my_model)

img_path = (jpg_name)
img = img_to_array(load_img(img_path, target_size=(50,50)))
img_nad = img_to_array(img)/255
img_nad = img_nad[None, ...]

label=['ota','TANAKA']
pred = model.predict(img_nad, batch_size=1, verbose=0)
score = np.max(pred)
pred_label = label[np.argmax(pred[0])]
print('name:',pred_label)
print('score:',score)

This time I tried to see how to distinguish when Hirate of Keyakizaka in a different file is put. スクリーンショット 2020-04-19 16.40.22.png

I recognized that it was Mr. Ota with a 99.9% probability lol スクリーンショット 2020-04-19 16.43.35.png

By the way, If you judge by the real Mr. Tanaka スクリーンショット 2020-04-19 16.46.01.png スクリーンショット 2020-04-19 16.47.40.png It will determine it properly! !! !!

This time, I tried to summarize the flow of image recognition using deep learning. I could not omit it as much as possible so that even beginners can do it. (I had a hard time, but lol) I think you can do it with your favorite actors and characters, so please come and join us! !!

Quote https://qiita.com/nirs_kd56/items/bc78bf2c3164a6da1ded

Recommended Posts

I tried face recognition of the laughter problem using Keras.
I tried face recognition using Face ++
I tried handwriting recognition of runes with CNN using Keras
I tried using the image filter of OpenCV
I tried using the trained model VGG16 of the deep learning library Keras
I tried using the API of the salmon data project
[Python] I tried to judge the member image of the idol group using Keras
I tried refactoring the CNN model of TensorFlow using TF-Slim
I tried using GrabCut of OpenCV
I tried face recognition from the video (OpenCV: python version)
[Python] I tried collecting data using the API of wikipedia
I tried face recognition with OpenCV
I tried using the checkio API
I tried to get the index of the list using the enumerate function
I looked at the meta information of BigQuery & tried using it
I tried the asynchronous server of Django 3.0
Face recognition of anime characters with Keras
Python: I tried the traveling salesman problem
I tried using the BigQuery Storage API
I tried to get the batting results of Hachinai using image processing
I tried to estimate the similarity of the question intent using gensim's Doc2Vec
I tried to extract and illustrate the stage of the story using COTOHA
I tried the common story of using Deep Learning to predict the Nikkei 225
Using COTOHA, I tried to follow the emotional course of Run, Melos!
I tried the common story of predicting the Nikkei 225 using deep learning (backtest)
I tried handwriting recognition of runes with scikit-learn
I tried using scrapy for the first time
I tried the pivot table function of pandas
I tried cluster analysis of the weather map
I tried image recognition of CIFAR-10 with Keras-Learning-
vprof --I tried using the profiler for Python
I tried image recognition of CIFAR-10 with Keras-Image recognition-
I solved the deepest problem of Hiroshi Yuki.
I tried using PyCaret at the fastest speed
I tried using the Google Cloud Vision API
I tried to touch the API of ebay
I tried to correct the keystone of the image
I tried using the Datetime module by Python
I tried using the functional programming library toolz
I tried to implement the traveling salesman problem
I tried to predict the price of ETF
I tried to vectorize the lyrics of Hinatazaka46!
I tried to predict the deterioration of the lithium ion battery using the Qore SDK
I tried to notify the update of "Hamelin" using "Beautiful Soup" and "IFTTT"
I tried to automate the face hiding work of the coordination image for wear
I tried using parameterized
I tried using argparse
I tried using mimesis
I tried using anytree
I tried using Summpy
I tried using coturn
I tried using Pipenv
I tried using matplotlib
I tried using "Anvil".
I tried using Hubot
I tried using ESPCN
I tried using openpyxl
I tried using Ipython
I tried using PyCaret
I tried using cron
I tried using ngrok