Introduction

I used to want to do image recognition with my own data instead of MNIST, but this is pretty good! !! There aren't many articles like that, and even if I could, the only output was the correct answer rate, and I couldn't experiment, so I'd like to summarize it. If I can afford it, I would like to write up to the point of making an image recognition application by installing it in a web application.

I think there are some things that cannot be reached, but I would appreciate it if you could point out any mistakes.

Rough flow, what you can do

We use CNN, which is the basis of recent image recognition methods. Create a model that can learn and recognize the image prepared by yourself, output the trained data, and read it with another code so that you can play with it. This time, I will create a program to distinguish between calico cats and white cats at my home.

environment

win10
python 3.7.3
numpy
Pillow 6.1.0
tensorflow==1.13.1
keras==2.1.2
sklearn

code

All the code is also available on github and you can download it from there.

Part 1: Data preparation & padding

First, prepare the data. It's very simple here, you just save the collected images in folders. (Separate the files according to what you want to distinguish here. The siro and mike in the image below are the same.)

Also, create a test_img folder to save the images you want to predict. スクリーンショット (94).png

Next, we will inflate the data and create a dataset. Inflating data means that it is very difficult to collect enough data by hand (it is troublesome), so once you have collected some images, you can rotate or invert the images to inflate the data set and learn. It will be a device to do. The code below is aug.py in the image above.

`aug.py`


from PIL import Image
import os, glob
import numpy as np
from keras.utils import np_utils
from sklearn import model_selection
from sklearn.model_selection import train_test_split

classes = ["mike","siro"]####Enter the label you want to identify here
num_classes = len(classes)
image_size = 128


#dataset directory
#The mike I made earlier,Enter the path to just before the siro folder
datadir='./'

#Loading images
X = []
Y = []


for index, classlabel in enumerate(classes):
    photos_dir = datadir+ classlabel
    files = glob.glob(photos_dir + "/*.jpg ")
    for i, file in enumerate(files):
        
        image = Image.open(file)
        image = image.convert("RGB")
        image = image.resize((image_size, image_size))
        #image.save("./test/{}{}.jpg ".format(classlabel,i))
        data = np.asarray(image)
        
        for angle in range(-20, 20, 5):##5
            #rotation
            img_r = image.rotate(angle)
            data = np.asarray(img_r)
            X.append(data)
            Y.append(index)

            #Invert
            img_trans = image.transpose(Image.FLIP_LEFT_RIGHT)
            data = np.asarray(img_trans)
            X.append(data)
            Y.append(index)



X = np.array(X)
Y = np.array(Y)

#To 20% test data
(X_train, X_test, y_train, y_test) = train_test_split(X, Y,test_size=0.2)

#Normalization
X_train = X_train.astype("float") / 255
X_test = X_test.astype("float") / 255

#Convert teacher data type
y_train = np_utils.to_categorical(y_train,num_classes)
y_test = np_utils.to_categorical(y_test, num_classes)

#X_train, X_test, y_train, y_test = model_selection.train_test_split(X, Y)
xy = (X_train, X_test, y_train, y_test)
np.save("./dataset.npy", xy)

When it finishes running, you should have a file called dataset.npy in the same directory. This is the dataset used this time.

Part 2: Learning

Read the dataset you just created and finally learn. The code below corresponds to CNN.py in the first image.

`CNN.py`



import keras
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.convolutional import Conv2D, MaxPooling2D
from keras.layers.core import Dense, Dropout, Activation, Flatten
from keras.layers import BatchNormalization
from keras.optimizers import SGD
import numpy as np
from sklearn.model_selection import train_test_split
from PIL import Image
import glob

from PIL import Image
import matplotlib.pyplot as plt

import os
from keras.callbacks import TensorBoard,ModelCheckpoint



#Hyperparameters
hp1 = {}
hp1['class_num'] = 2 #Number of classes (this time mike,Because it is 2 classes of siro 2)
hp1['batch_size'] = 64 #Batch size#####32
hp1['epoch'] = 20 #Number of epochs



#Dataset loading
##Load the dataset created in the previous chapter here
X_train, X_test, y_train, y_test = np.load("./dataset.npy", allow_pickle=True)


#Input size
input_shape=X_train.shape[1:]

#Build CNN
def CNN(input_shape):
        model = Sequential()
 
        model.add(Conv2D(32, (3, 3), padding='same',input_shape=input_shape))
        model.add(Activation('relu'))
        model.add(Conv2D(32, (3, 3)))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(MaxPooling2D(pool_size=(2, 2)))
        model.add(Dropout(0.25))

        model.add(Conv2D(64, (3, 3), padding='same'))
        model.add(Activation('relu'))
        model.add(Conv2D(64, (3, 3)))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(MaxPooling2D(pool_size=(2, 2)))
        model.add(Dropout(0.25))

        model.add(Conv2D(128, (3, 3), padding='same'))
        model.add(Activation('relu'))
        model.add(Conv2D(128, (3, 3)))
        model.add(BatchNormalization())
        model.add(Activation('relu'))

        model.add(MaxPooling2D(pool_size=(2, 2)))
        model.add(Dropout(0.25))


        model.add(Flatten())
        model.add(Dense(512))
        model.add(BatchNormalization())
        model.add(Activation('relu'))
        model.add(Dropout(0.5))
        model.add(Dense(hp1['class_num']))
        model.add(Activation('softmax'))
    
        return model
 

#Select a model
model=CNN(input_shape)



#compile

model.compile(loss='categorical_crossentropy',optimizer='SGD',metrics=['accuracy'])


#Data recording
log_dir = os.path.join(os.path.dirname(__file__), "logdir")
model_file_name="model_file.hdf5"

#Training
history = model.fit(
        X_train, y_train,
         epochs=hp1['epoch'],
         validation_split = 0.2,
         callbacks=[
                TensorBoard(log_dir=log_dir),
                ModelCheckpoint(os.path.join(log_dir,model_file_name),save_best_only=True)
                ]
        )

#Evaluation&Evaluation result output


loss,accuracy = model.evaluate(X_test, y_test, batch_size=hp1['batch_size'])

It will take some time to learn here. I think it's a messed up warning, but don't worry ... When the execution is finished, a folder called logdir will be created, and there should be a file called ** model_file.hdf5 ** in it. This is the most important and trained model.

Part 3: Predict recognition results and play

Finally play with the trained model. I think the folder looks like this now.

スクリーンショット (96).png Please put the image you want to predict in this test_img,

Read the ** model_file.hdf5 ** output earlier and make a prediction. The following code will be predict.py.

`predict.py`


from pathlib import Path
import numpy as np
from PIL import Image
from keras.models import load_model
import glob

#####  mike=0,siro=1  ######


###Loading trained model
model_path = "./logdir/model_file.hdf5"

##The location of the data you want to predict
images_folder = "./test_img"

classes = ["mike","siro"]

# load model
model = load_model(model_path)

#image_size=100
image_size=128
X = []


dir = images_folder
#Check the path
#print(dir)

files = glob.glob(dir + "/*.jpg ")
for i, file in enumerate(files):
    image = Image.open(file)
    image = image.convert("RGB")
    image = image.resize((image_size, image_size))
    data = np.asarray(image)
    X.append(data)
    
 
X = np.array(X)


#Normalization(0-1)
X = X.astype('float32')
X = X / 255.0

#print(len(files))

##softmax
for w in range(len(files)):

    result = model.predict([X])[w]
    predicted = result.argmax()
    percentage = int(result[predicted] * 100)
    print(files[w].split('\\')[-1])
    print("{0}({1} %)".format(classes[predicted],percentage))

This time I tried to input such an image. (cute) c5d7411c6bd88106ebab0730b8af4f907_4620693218538323992_200325_0003.jpg

I think the execution result of predict.py looks like this. スクリーンショット (98).png

You can predict that it is a white cat safely! !!

Summary

How was that? ?? I hope you can play safely. With the development from here, if you want to make an image recognition application, you can create an application that returns predictions if you have the trained models model_file.hdf5 and predict.py on the server.

For reference, the image recognition web application that I made at the hackathon and other members in the past and presented at the technical education exhibition is here

References

--Deep Learning from scratch --Intuition Deep Learning

Let's make an image recognition model with your own data and play!