I tried CNN fine tuning with Resnet

Fine tuning with Python Resnet

** This article introduces fine tuning using Resnet. ** **

This time, in order to verify how effective fine tuning is, it is normal. I would like to compare [Model created by CNN] and [Fine-tuned model].

Development environment

I learned with Google Colab, but I will describe it as possible even with local assumptions. python : 3.7.0 keras : 2.4.3 tensorflow : 2.2.0

What is Resnet

Resnet is a [learned] convolutional neural network with over 1 million images in a database called Imagenet. And this network is also called Resnet 50, which has 50 layers and can classify 1000 categories.

What is fine tuning?

Retraining the weights of the entire model, with the trained network weights as the initial value. Therefore, after using the above Resnet50, we aim to make a better discriminator by re-learning.

Image to use

It seemed interesting, so I tried using 3 types of Japanese cigarettes. ** Mobius: 338 sheets ** ** Seven Stars: 552 sheets ** ** Winston: 436 sheets **

This time it is a verification of fine tuning, so the above image was pulled from the Internet.

Normal CNN

The configuration is as follows. 1.png

Learning this resulted in the following results. tobacco_dataset_reslut.jpg

It's pretty bad because we haven't done any pre-processing such as cropping the image. Is it around [60%] in the test data?

Source

normal_cnn.py


from PIL import Image
import numpy as np
import glob
import os
from keras.utils import np_utils
from keras.models import Sequential
from keras.layers.convolutional import MaxPooling2D
from keras.layers import  Conv2D, Flatten, Dense, Dropout
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt

root = "tobacco_dataset"
folder = os.listdir(root)
image_size = 224
dense_size = len(folder)
epochs = 30
batch_size = 16

X = []
Y = []
for index, name in enumerate(folder):
    dir = "./" + root + "/" + name
    print("dir : ", dir)
    files = glob.glob(dir + "/*")
    print("number : " + str(files.__len__()))
    for i, file in enumerate(files):
      try:
        image = Image.open(file)
        image = image.convert("RGB")
        image = image.resize((image_size, image_size))
        data = np.asarray(image)
        X.append(data)
        Y.append(index)
      except :
          print("read image error")

X = np.array(X)
Y = np.array(Y)
X = X.astype('float32')
X = X / 255.0

Y = np_utils.to_categorical(Y, dense_size)
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.15)

model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(image_size, image_size, 3)))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(dense_size, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.summary()

result = model.fit(X_train, y_train, validation_split=0.15, epochs=epochs, batch_size=batch_size)

x = range(epochs)
plt.title('Model accuracy')
plt.plot(x, result.history['accuracy'], label='accuracy')
plt.plot(x, result.history['val_accuracy'], label='val_accuracy')
plt.xlabel('Epoch')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5), borderaxespad=0, ncol=2)

name = 'tobacco_dataset_reslut.jpg'
plt.savefig(name, bbox_inches='tight')
plt.close()


Fine tuning with Resnet

Since the composition is quite deep and can not be pasted on Qiita, I will describe the URL https://github.com/daichimizuno/cnn_fine_tuning/blob/master/finetuning_layer.txt

In addition to the original Resnet, the activation function Relu and dropout are added by 0.5. By the way, the following sources are the sources when using Resnet. It seems that Keras contains a library for Resnet.


ResNet50 = ResNet50(include_top=False, weights='imagenet',input_tensor=input_tensor)

Learning this resulted in the following results. resnet_tobacco_dataset_reslut.jpg

Is it around [85%] in the test data?

Source

resnet_fine_tuning.py


from PIL import Image
import numpy as np
import glob
import os
from keras.utils import np_utils
from keras.models import Sequential, Model
from keras.layers import Flatten, Dense,Input, Dropout
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
from keras.applications.resnet50 import ResNet50
from keras import optimizers

root = "tobacco_dataset"
folder = os.listdir(root)
image_size = 224
dense_size = len(folder)
epochs = 30
batch_size = 16

X = []
Y = []
for index, name in enumerate(folder):
    dir = "./" + root + "/" + name
    print("dir : ", dir)
    files = glob.glob(dir + "/*")
    print("number : " + str(files.__len__()))
    for i, file in enumerate(files):
      try:
        image = Image.open(file)
        image = image.convert("RGB")
        image = image.resize((image_size, image_size))
        data = np.asarray(image)
        X.append(data)
        Y.append(index)
      except :
          print("read image error")

X = np.array(X)
Y = np.array(Y)
X = X.astype('float32')
X = X / 255.0

Y = np_utils.to_categorical(Y, dense_size)
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.15)

input_tensor = Input(shape=(image_size, image_size, 3))
ResNet50 = ResNet50(include_top=False, weights='imagenet',input_tensor=input_tensor)

top_model = Sequential()
top_model.add(Flatten(input_shape=ResNet50.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(dense_size, activation='softmax'))

top_model = Model(input=ResNet50.input, output=top_model(ResNet50.output))
top_model.compile(loss='categorical_crossentropy',optimizer=optimizers.SGD(lr=1e-3, momentum=0.9),metrics=['accuracy'])

top_model.summary()
result = top_model.fit(X_train, y_train, validation_split=0.15, epochs=epochs, batch_size=batch_size)

x = range(epochs)
plt.title('Model accuracy')
plt.plot(x, result.history['accuracy'], label='accuracy')
plt.plot(x, result.history['val_accuracy'], label='val_accuracy')
plt.xlabel('Epoch')
plt.legend(loc='center left', bbox_to_anchor=(1, 0.5), borderaxespad=0, ncol=2)

name = 'resnet_tobacco_dataset_reslut.jpg'
plt.savefig(name, bbox_inches='tight')
plt.close()


Summary

Comparing normal CNN and fine tuning, you can see that fine tuning has improved sharply from around Epoch 10. Since this is re-learning using Resnet, I imagine that it will propagate in the part where the correct answer rate of Resnet increases and affect tobacco discrimination, but more detailed analysis is necessary. ..

However, in any case, the result was considerably higher than that of normal CNN in terms of tobacco judgment. I think that even better results will be obtained if you select images and perform pre-processing, so if you can, please try it!

【Github】 https://github.com/daichimizuno/cnn_fine_tuning

Please point out any mistakes or unclear points. that's all

Recommended Posts

I tried CNN fine tuning with Resnet
I tried fp-growth with python
I tried scraping with Python
I tried Learning-to-Rank with Elasticsearch!
I tried clustering with PyCaret
[MNIST] I tried Fine Tuning using the ImageNet model.
I tried gRPC with Python
I tried to implement Harry Potter sort hat with CNN
I tried trimming efficiently with OpenCV
I tried summarizing sentences with summpy
I tried machine learning with liblinear
I tried web scraping with python.
I tried moving food with SinGAN
I tried implementing DeepPose with PyTorch
Various Fine Tuning with Mobilenet v2
I tried running prolog with python 3.8.2.
I tried SMTP communication with Python
I tried sentence generation with GPT-2
I tried learning LightGBM with Yellowbrick
I tried face recognition with OpenCV
I tried sending an SMS with Twilio
I tried linebot with flask (anaconda) + heroku
I tried to visualize AutoEncoder with TensorFlow
I tried to get started with Hy
I tried factor analysis with Titanic data!
I tried learning with Kaggle's Titanic (kaggle②)
I tried sending an email with python.
I tried non-photorealistic rendering with Python + opencv
I tried a functional language with Python
I tried batch normalization with PyTorch (+ note)
I tried implementing DeepPose with PyTorch PartⅡ
I tried to implement CVAE with PyTorch
I tried playing with the image with Pillow
I tried to solve TSP with QAOA
I tried simple image recognition with Jupyter
I tried natural language processing with transformers.
#I tried something like Vlookup with Python # 2
765 I tried to identify the three professional families by CNN (with Chainer 2.0.0)
I tried handwriting recognition of runes with scikit-learn
I tried to predict next year with AI
I tried "smoothing" the image with Python + OpenCV
I tried hundreds of millions of SQLite with python
I tried to use lightGBM, xgboost with Boruta
I tried image recognition of CIFAR-10 with Keras-Learning-
I tried to learn logical operations with TF Learn
I tried to move GAN (mnist) with keras
I tried "differentiating" the image with Python + OpenCV
I tried scraping
I tried "License OCR" with Google Vision API
I tried to save the data with discord
I tried PyQ
I tried to integrate with Keras in TFv1.1
I tried Flask with Remote-Containers of VS Code
I tried non-negative matrix factorization (NMF) with TensorFlow
I tried L-Chika with Raspberry Pi 4 (Python edition)
I tried Python-like loop processing with BigQuery Scripting
I tried playing with PartiQL and MongoDB connected
I tried principal component analysis with Titanic data!
I tried Jacobian and partial differential with python
I tried to get CloudWatch data with Python
I tried using mecab with python2.7, ruby2.3, php7