I tried to use deep learning to extract the part where the plant is shown from the photo of the balcony, but it didn't work, so I will summarize the contents of trial and error. Part 2

Last synopsis

A stock that started with a little effort. However, I never imagined that a new coronavirus infection would prevail, and I probably failed to make an investment without omission, and the savings I had saved so far to purchase my home became empty, and I became desperate and desperate. When I was walking along the lake bridge, I was told from behind, "It's been a while. How are you ?!" ...

... that's a joke, and I'll continue the article about the theme of extracting objects from photos!

I tried to use deep learning to extract the part of the balcony where the plants are shown, but it didn't work, so I'll summarize the contents of trial and error. Part 1

Consideration

After the first part, I tried changing some parameters, but it didn't work at all, and only empty time passed ...

Meanwhile, I suddenly came up with "transfer learning." It is a method that realizes highly accurate identification from a small amount of data using the output of the trained model ...

I don't know how to do it at all, but for the time being, I decided to try it with my knowledge!

Learning

Read the same data as the first part, let the trained model identify it, and let the neural network train the result as input.

python


from glob import glob
import random

X = []
y = []

dirs = ["9*/*.jpg ", "0*/*.jpg "]

i = 0
min = 1500

for d in dirs:
    files = glob(d)
    files = random.sample(files, min if len(files) > min else len(files))
    
    for f in files:
        X.append(f)
        y.append(i)

    i += 1
    
import pandas as pd

df = pd.DataFrame({"X" : X, "y" : y})

from keras.applications.vgg16 import VGG16, preprocess_input, decode_predictions

#Get the trained model of VGG16
model_VGG16 = VGG16()

import cv2
import numpy as np
from keras.preprocessing import image

X = []
y = []

cnt = 0

for idx in df.index:

    f = df.loc[idx, "X"]
    l = df.loc[idx, "y"]

    print("\r{:05} : [{}] {}".format(cnt, l, f), end="")
    i += 1

    #Loading images
    img = image.load_img(f, target_size=model_VGG16.input_shape[1:3])

    X.append(image.img_to_array(img))
    y.append(l)

    cnt += 1

print()

#Preprocessing
X = np.array(X)
X = preprocess_input(X)

#Identification by VGG16
preds = model_VGG16.predict(X)
print('preds.shape: {}'.format(preds.shape))  # preds.shape: (1, 1000)

#Set the identification result as input
X = preds

from sklearn import model_selection

test_size = 0.2

#Split for learning, validation and testing
X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=test_size, random_state=42)
X_valid, X_test, y_valid, y_test = model_selection.train_test_split(X_test, y_test, test_size=.5, random_state=42)

from keras.utils.np_utils import to_categorical

y_train = to_categorical(y_train)
y_valid = to_categorical(y_valid)
y_test = to_categorical(y_test)

from keras.layers import Dense
from keras.models import Sequential

#Creating a learning model
model = Sequential()

model.add(Dense(64, input_dim=len(X_train[0]), activation='relu'))
model.add(Dense(64, activation='sigmoid'))

model.add(Dense(len(y_train[0]), activation='softmax'))

#compile
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

#Learning
history = model.fit(
    X_train, y_train, batch_size=25, epochs=60, 
    verbose=1, shuffle=True,
    validation_data=(X_valid, y_valid))

import matplotlib.pyplot as plt

#Evaluation and display of generalization system
score = model.evaluate(X_test, y_test, batch_size=32, verbose=0)
print('validation loss:{0[0]}\nvalidation accuracy:{0[1]}'.format(score))

#acc, val_acc plot
plt.plot(history.history["accuracy"], label="acc", ls="-", marker="o")
plt.plot(history.history["val_accuracy"], label="val_acc", ls="-", marker="x")
plt.ylabel("accuracy")
plt.xlabel("epoch")
plt.legend(loc="best")
plt.show()

#loss, val_loss plot
plt.plot(history.history["loss"], label="loss", ls="-", marker="o")
plt.plot(history.history["val_loss"], label="val_loss", ls="-", marker="x")
plt.ylabel("loss")
plt.xlabel("epoch")
plt.legend(loc="best")
plt.show()

Run! !!

validation loss:0.0818712607031756
validation accuracy:0.9797570705413818

Unknown1.pngUnknown2.png

Oh! what's this? !!

Identification and marking

python


import cv2
import numpy as np

div = 10
cl = (0, 0, 255)

i = 0

#Get a list of test images
testFiles = glob("test/*.JPG")
testFiles.sort()

for f in testFiles:

    print(f.split("/")[-1])

    #Read file
    img = cv2.imread(f)
    img_output = cv2.imread(f)
    #print(img.shape)

    h, w, c = img.shape
    print(f, h, w, c)

    #Get short edges and set split size
    size = h if h < w else w
    size = int(size / 10)
    print(size)

    i = 0
    k = 0

    while size * (i + 1) < h:
        j = 0
        while size * (j + 1) < w:

            #Get split image
            img_x = img[size*i:size*(i+1), size*j:size*(j+1)]
            
            #Identification by VGG16
            img_test = cv2.resize(img_x, model_VGG16.input_shape[1:3])
            img_test = cv2.cvtColor(img_test, cv2.COLOR_BGR2RGB)
            X = np.array([img_test])
            X = preprocess_input(X)
            preds = model_VGG16.predict(X)

            #identification
            pred = model.predict(preds, batch_size=32)
            m = np.argmax(pred[0])

            #Accuracy is 85%When it is identified as a plant
            if pred[0][m] > 0.85 and m == 1:
                #Draw a red square where it is identified as a plant
                cv2.rectangle(img_output, (size*j + 5, size*i + 5), (size*(j+1) - 5, size*(i+1) - 5), cl, thickness=2)

            j += 1
            k += 1

        i += 1
    
    img_output = img_output[0:size*i, 0:size*j]

    f_new = "predicted/{}".format(f.split("/")[-1])
    cv2.imwrite(f_new, img_output)

    print()

Run! !!

IMG_0791.JPG IMG_0805.JPG IMG_0826_転移学習_EPOC100.JPG

Perfect! !!

Transfer learning is amazing! ... may be different from the transfer learning in the original sense. Sweat

By the way, I didn't mark it by hand. Lol

Well, what will we do next?

Recommended Posts

I tried to use deep learning to extract the part where the plant is shown from the photo of the balcony, but it didn't work, so I will summarize the contents of trial and error. Part 2
I didn't understand the Resize of TensorFlow so I tried to summarize it visually.
The tree.plot_tree of scikit-learn was very easy and convenient, so I tried to summarize how to use it easily.
I didn't understand the behavior of numpy's argsort, so I will summarize it.
I tried to use Java with Termux using Termux Arch but it didn't work
I tried to use Twitter Scraper on AWS Lambda and it didn't work.
[Part 4] Use Deep Learning to forecast the weather from weather images
[Part 1] Use Deep Learning to forecast the weather from weather images
[Part 3] Use Deep Learning to forecast the weather from weather images
[Part 2] Use Deep Learning to forecast the weather from weather images
[Machine learning] I tried to summarize the theory of Adaboost
I tried to implement Perceptron Part 1 [Deep Learning from scratch]
[Qiita API] [Statistics • Machine learning] I tried to summarize and analyze the articles posted so far.
[Deep Learning from scratch] I tried to implement sigmoid layer and Relu layer.
I installed PySide2, but pyside2-uic didn't work, so I managed to do it.
I tried to install Docker on Windows 10 Home but it didn't work
I tried to extract and illustrate the stage of the story using COTOHA
I tried the common story of using Deep Learning to predict the Nikkei 225
I wanted to use the find module of Ansible2, but it took some time, so make a note
Use Pillow to make the image transparent and overlay only part of it
I tried to extract a line art from an image with Deep Learning
Docker x visualization didn't work and I was addicted to it, so I summarized it!
I tried to classify Oba Hana and Emiri Otani by deep learning (Part 2)
I tried my best to make an optimization function, but it didn't work.
Judgment whether it is my child from the photograph of Shiba Inu by deep learning (4) Visualization by Grad-CAM and Guided Grad-CAM