I made an app to find out who the members of the Straw Hat Pirates look like

I made an app to find out who the members of the Straw Hat Pirates look like

This time, I made an application that uses image recognition of machine learning. As the title suggests, you can find out how much your face resembles who is a member of the One Piece Straw Hat Pirates.

Click here for git hub ↓↓↓ https://github.com/Sho-cmyk/shiny-winner

First, look at the completed form. 図1.png When you press the shooting button, It shows how much your face looks like to any member of the Straw Hat Pirates. The top two similar characters are displayed. ャ.JPG I found that it was 38% similar to Jinbei and 20% similar to Luffy.

If you are interested in the app, please try it from this URL. ↓↓↓ https://one-piece-camera.herokuapp.com/

Steps to make an app

Next, I will introduce the procedure for making an application. The procedure is roughly divided as follows.

  1. File structure (flask)

    1. Collect images for learning
  2. Image preprocessing

    1. Learning
  3. Estimate

  4. Arrange the output screen with html / css / js

  5. File structure in Flask

flask_app
├ one-piece_cnn_aug.h5 ---Save weight
├ static ---Static file location
│ ├ image
│ └ webcam.js
│ └ stylesheet.css
├ templates ---template
│ └ index.html
│ └ photoclick.html
│ └ photoclick_copy.html
├ main.py ---Application execution script (application entrance)
├ requirements.txt 
└ runtime.txt
└ Profile
└ README.md
    1. Collect images for learning I use Yahoo image search to scrape images of each member of the Straw Hat Pirates. I scraped 200 sheets each. For the detailed scraping method, refer to the following. Quote: https://www.youtube.com/watch?v=BrCaYYJtC4w
  1. Image preprocessing I will leave only the images that can be used next. I will erase the photos that the target character has not moved by itself. 図2.png

The image is now divided into training data and test data. Training data is inflated using the Image Data Generator.

generatedata.py


from PIL import Image
import os,glob
import numpy as np
from sklearn.model_selection import train_test_split
from keras.preprocessing.image import ImageDataGenerator

classes=["Usopp","Sanji","Jinbei","Zoro","chopper","Nami","Bibi","Frankie","Brook","Luffy","Robin"]
num_classes=len(classes)
image_size=50
num_testdata=30

#Loading images
X_train=[]
X_test=[]
Y_train=[]
Y_test=[]

for index,classlabel in enumerate(classes):
    photos_dir="./"+classlabel
    files=glob.glob(photos_dir+"/*.jpg ")
    for i,file in enumerate(files):
        if i >=200:break
        image= Image.open(file)
        image=image.convert("RGB")
        image=image.resize((image_size,image_size))
        data=np.asarray(image)
        X_train.append(data) #Add to the end of the list
        Y_train.append(index) #Add label
#Standardization
datagen=ImageDataGenerator(samplewise_center=True,samplewise_std_normalization=True)


g=datagen.flow(X_train,Y_train,shuffle=False)
X_batch,y_batch=g.next()

#0 while maintaining the ratio of rgb values-Close to 255 for easy viewing
X_batch *=127.0/max(abs(X_batch.min()),X_batch.max())
X_batch +=127.0
X_batch =X_batch.astype('unit8')

#Put it in an array and divide it into test and training data
X_train=np.array(X_train)
X_test=np.array(X_test)
y_train=np.array(Y_train)
y_test=np.array(Y_test)

#X_train,X_test,y_train,y_test=train_test_split(X,Y)
xy=(X_train,X_test,y_train,y_test)
np.save("./one-piece_aug.npy",xy)#Save code

    1. Learning The learning model uses CNN. The code is below.

one-piece_cnn_aug.py


from keras.models import Sequential
from keras.layers import Conv2D,MaxPooling2D
from keras.layers import Activation,Dropout,Flatten,Dense
from keras.utils import np_utils
import keras
import numpy as np
from tensorflow.keras.optimizers import RMSprop

classes=["Usopp","Sanji","Jinbei","Zoro","chopper","Nami","Bibi","Frankie","Brook","Luffy","Robin"]
num_classes=len(classes)
image_size=50
#Define the main function that performs the training
def main():
    X_train,X_test,y_train,y_test=np.load("./one-piece.npy",allow_pickle=True) #Read data from a text file
    X_train=X_train.astype("float")/256 #Normalizes and converges RGB values to 0 to 1 to make neural networks easier to learn
    X_test=X_test.astype("float")/256 #Also
    y_train=np_utils.to_categorical(y_train,num_classes) #one-hot-vector:The correct answer value is 1, and the others are 0. The first argument is the label you want to convert to a vector, and the second argument is the label you can convert to a vector.
    y_test=np_utils.to_categorical(y_test,num_classes) #Also

    model=model_train(X_train,y_train) #Do learning
    model_eval(model,X_test,y_test) #Make an evaluation

#Define a model for CNN
def model_train(X,y):
    model=Sequential() #Create a Sequential model that connects each layer of the neural network in order
    #Take out features
    model.add(Conv2D(32,(3,3),padding='same',input_shape=X.shape[1:])) #32 layers of convolution layer 3x3
    model.add(Activation('relu')) #Activation function relu,Negative → 0, positive → as is
    model.add(Conv2D(32,(3,3))) #2nd convolution layer
    model.add(Activation('relu')) #Activation function relu
    model.add(MaxPooling2D(pool_size=(2,2))) #The pooling layer makes the features stand out and takes out
    model.add(Dropout(0.25)) #Discard 25% of data to reduce data bias

    model.add(Conv2D(64,(3,3),padding='same')) #Next is 64 kernel layers
    model.add(Activation('relu')) #relu activation function
    model.add(Conv2D(64,(3,3))) #Also a convolution layer
    model.add(Activation('relu')) #relu activation function
    model.add(MaxPooling2D(pool_size=(2,2))) #Max pooling

    #Fully join. Classify based on characteristics
    model.add(Flatten()) #First, arrange the data in a row
    model.add(Dense(512)) #Fully join
    model.add(Activation('relu')) #relu activation function
    model.add(Dropout(0.5)) #Discard half
    model.add(Dense(11)) #11 output layer nodes
    model.add(Activation('softmax')) #Use a softmax function that sums up to 1

    opt=RMSprop(lr=0.0001,decay=1e-6) #Definition of optimization method. Update algorithm during training.

    model.compile(loss='categorical_crossentropy',
                    optimizer=opt,metrics=['accuracy']) #Evaluation function. loss:The error between the correct answer and the estimated value in the loss function. opt:Optimization method. metrics:Evaluation value

    model.fit(X,y,batch_size=32,epochs=100) #Parameter optimization. batch_size:The number of data used at one time. epoch: Number of learnings.
#Save model
    model.save('./one-piece_cnn.h5')

    return model

#Make the model usable
def model_eval(model,X,y):
    scores=model.evaluate(X,y,verbose=1) #verbose:Show progress,Put the learning result in score
    print('Test Loss:',scores[0]) #loss
    print('Test Accuracy:',scores[1]) #Accuracy

#Allow this program to be called by other files
if __name__=="__main__":
    main()

The learning results are as follows.

Epoch 100/100
7888/7888 [==============================] - 64s 8ms/step - loss: 0.0200 - acc: 0.9962
330/330 [==============================] - 1s 4ms/step
  1. Estimate

main.py


import PIL
import io
import base64
import re
from io import StringIO
from PIL import Image
import os
from flask import Flask, request, redirect, url_for, render_template, flash,jsonify
from werkzeug.utils import secure_filename
from keras.models import Sequential, load_model
from keras.preprocessing import image
import tensorflow as tf
import numpy as np
from datetime import datetime

classes=["Usopp","Sanji","Jinbei","Zoro","chopper","Nami","Bibi","Frankie","Brook","Luffy","Robin"]
classes_img=["static/img/usoppu.jpg ","static/img/sanji.jpg ","static/img/jinbe.jpg ","static/img/zoro.jpg ","static/img/chopper.jpg ","static/img/nami.jpg ","static/img/bibi.jpg ","static/img/franky.jpg ","static/img/bruck.jpg ","static/img/rufi.jpg ","static/img/robin.jpg "]
num_classes=len(classes)
image_size=50

UPLOAD_FOLDER = "./static/image/"
ALLOWED_EXTENSIONS = set(['png', 'jpg', 'jpeg', 'gif'])

app = Flask(__name__)

def allowed_file(filename):
    return '.' in filename and filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS

model = load_model('./one-piece_cnn_aug.h5')#Load the trained model

graph = tf.get_default_graph()


app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def upload_file():

    global graph
    with graph.as_default():
        if request.method == 'POST':

                myfile= request.form['snapShot'].split(',')
                imgdata = base64.b64decode(myfile[1])
                image = Image.open(io.BytesIO(imgdata))

                #Save
                basename = datetime.now().strftime("%Y%m%d-%H%M%S")
                image.save(os.path.join(UPLOAD_FOLDER, basename+".png "))
                filepath = os.path.join(UPLOAD_FOLDER, basename+".png ")


                image=image.convert('RGB')
                image=image.resize((image_size,image_size))
                data=np.asarray(image)
                X=[]
                X.append(data)
                X=np.array(X).astype('float32')
                X /=256

                result=model.predict([X])[0]
                result_proba = model.predict_proba(X, verbose=1)[0]
                    
                percentage=(result_proba*100).astype(float)
                array_sort =sorted(list(zip(percentage,classes,classes_img)),reverse=True)
                percentage,array_class,array_img = zip(*array_sort)
                
                
                pred_answer1="label:"+str(array_class[0])+ ",probability:"+str(percentage[0])+"%"
                pred_answer2="label:"+str(array_class[1])+ ",probability:"+str(percentage[1])+"%"
                pred_answer3="label:"+str(array_class[2])+ ",probability:"+str(percentage[2])+"%"
                
                img_src1=array_img[0]
                img_src2=array_img[1]
                img_src3=array_img[2]


                basename = datetime.now().strftime("%Y%m%d-%H%M%S")
                filepath3 = UPLOAD_FOLDER + basename+".png "


                return render_template("index.html",answer1=pred_answer1,img_data1=img_src1,answer2=pred_answer2,img_data2=img_src2,answer3=pred_answer3,img_data3=img_src3)

        return render_template("index.html",answer="")

        

if __name__ == '__main__':
   app.run(debug = True)


Five. Arrange the output screen All you have to do is arrange the output screen with html / css / js.

Recommended Posts

I made an app to find out who the members of the Straw Hat Pirates look like
I made an appdo command to execute a command in the context of the app
With LINEBot, I made an app that informs me of the "bus time"
I used Python to find out about the role choices of the 51 "Yachts" in the world.
I tried to find the entropy of the image with python
I tried to find out the outline about Big Gorilla
I made a function to check the model of DCGAN
(Python) I made an app from Trello that periodically notifies slack of tasks that are about to expire.
I made an npm package to get the ID of the IC card with Raspberry Pi and PaSoRi
How to find out the number of CPUs without using the sar command
I tried to find the optimal path of the dreamland by (quantum) annealing
I want to analyze the emotions of people who want to meet and tremble
I want to leave an arbitrary command in the command history of Shell
I made a program to check the size of a file in Python
I made a function to see the movement of a two-dimensional array (Python)
The end of catastrophic programming # 02 "Dice 5 and 6 are hard to come out ... I feel like random numbers are considered"
I made an online frequency analysis app
I made a tool to estimate the execution time of cron (+ PyPI debut)
I used gawk to find out the maximum value that goes into NF.
A programming beginner tried to find out the execution time of sorting etc.
I made you to express the end of the IP address with L Chika
I made a tool to automatically back up the metadata of the Salesforce organization
I want to store the result of% time, %% time, etc. in an object (variable)
I made a program to look up words on the window (previous development)
I made a script to record the active window using win32gui of Python