[MNIST] I tried Fine Tuning using the ImageNet model.

① Introduction

When studying AI, I would like to study ** Fine Tuning ** as well. This time, I would like to implement ** MNIST ** using ** ImageNet trained DenseNet121 model **. The library uses ** keras **.

ai_study_kikaigakusyu.png

② What is fine tuning?

** Fine Tuning ** means ** fine-tuning ** when translated into Japanese. ** Fine-tuning ** in AI is a method of reusing a part of an existing model to build a new model.

■ Difference between fine tuning and transfer learning

Similar to ** Fine Tuning ** is ** Transfer Learning **. ** Fine tuning ** and ** Transfer learning ** are always messed up, so I'll organize them here.

-** Fine tuning ** ** Fine tuning ** trains again with the weight of the trained model as the initial value. The merit is that the weight of the existing model can be additionally learned as the initial value, so that the model can be constructed more efficiently and with high accuracy.

-** Transfer learning ** ** Transfer learning ** fixes the weights of the trained model and trains using only the added layers. The merit is that additional training is performed based on the trained model, so the model can be built with less data and in a short time.

■ So which one is this time?

I would like to reuse the image classification model that has already been completed to classify numbers. This time, I would like to use the "model of image classification with weights learned by ImageNet" to "classify numerical images". So, this time it will be ** fine tuning **.

③ Fine tuning method

I would like to use DenseNet121 for the model that is the ** fine tuning source **. This model is a model that is lightweight but has a good recognition rate.

** MNIST training data ** is ** 28 x 28 x 1ch **, ** DenseNet121 input layer ** is ** 32 or more x 32 or more x 3ch ** , These need to be combined. There are two ways to match.

** ・ How to match MNIST images ** → How to change the learning image to 32 or more x 32 or more x 3ch according to the input layer of the model

**-How to change the input layer of the model ** → How to change the input layer of the model to 1ch according to the learning image

■ How to match MNIST images

The input format of DenseNet121 is (height = 32 or more, width = 32 or more, channels = 3ch). The image format of MNIST is (height = 28, width = 28, channels = 1ch). The training image size is 32 or more x 32 or more x3ch is required from the network definition of DenseNet121, so perform the following processing.

--Resize image to 64x64 --Convert from grayscale 1ch to RGB3ch

001.png

I will explain using snippets in ④

■ How to change the input layer of the model

The input format of DenseNet121 is (height = 32 or more, width = 32 or more, channels = 3ch). The image format of MNIST is (height = 28, width = 28, channels = 1ch). The training image size is 32 or more x 32 or more x3ch is required from the network definition of DenseNet121, so perform the following processing.

--Resize image to 64x64 --Added 1ch input layer

002.png

I will explain using snippets in ⑤

④ How to match MNIST images

Basically, fine tuning can be performed by following the steps ➊ to ➎ below.

➊ Download learned weights

With the API of " keras.applications.densenet.DenseNet121 ", you can easily ** download the weights trained by imagenet with Densenet121 **. It's hard to download every time (33MB), so save it in Google Drive.

from keras.applications.densenet import *

model = DenseNet121(weights='imagenet')
model.save(fullpath)

➋ MNIST data conversion

Converts MNIST images from (28, 28, 1) to (64, 64, 3) according to the input format (height, width, channels) of DenseNet121.

import cv2
import numpy as  np
from keras.datasets import mnist

img_w, img_h, img_ch = 64, 64, 3
num_classes = 10

def pretreatment(x):
    #Resize image
    X = []
    for i in range(len(x)):
        dst = cv2.resize(x[i], (img_h, img_w), interpolation=cv2.INTER_CUBIC)
        dst = cv2.cvtColor(dst, cv2.COLOR_GRAY2RGB)
        X.append(dst)

    #Reshape
    X = np.array(X).reshape(len(x), img_h, img_w, img_ch)

    #Change data in the range 0 to 255 to the range 0 to 1
    ret = X.astype('float32') / 255
    return ret

#■ Read MNIST data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_valid, y_train, y_valid = train_test_split(x_train, y_train, test_size=0.2)

#■ Learning(Train)data
x_train = pretreatment(x_train)
y_train = np_utils.to_categorical(y_train, num_classes)

#■ Learning(Valid)data
x_valid = pretreatment(x_valid)
y_valid = np_utils.to_categorical(y_valid, num_classes)

#■ Test data
x_test = pretreatment(x_test)
y_test = np_utils.to_categorical(y_test, num_classes)

➌ Build a network of DenseNet121

(1) DneseNet121 network construction

With keras, you can easily ** build a network of Densenet121 ** by using the API of " keras.applications.densenet.DenseNet121 ". By setting include_top to False, you can prevent the fully connected layer on the output layer side of the network from being included. Also, do not read weights at this point.

input_tensor = x_train.shape[1:]
model = DenseNet121(include_top=False, weights=None, input_shape=input_tensor)

(2) Addition of output layer

Build a fully connected layer on the output layer side of the network DenseNet121 (ImageNet) discriminates 1000 classes as image classification, but since this time we will implement MNIST, we will make it possible to classify into 10 classifications from 0 to 9.

from keras.layers import *
from keras.models import Model

x = Flatten()(model.layers[-1].output)
x = Dense(10, activation="softmax")(x)
Model(model.inputs, x)

➍ Read learned weights

After configuring the network, load the ** weights downloaded in ➊ **. The weights are loaded by this " load_weights "method.

model.load_weights(fullpath, by_name=True)

➎ Learn

After that, you can do ** Fine Tuning ** by learning as usual.

➏ Execution result

● TEST: Verification result
Test loss    : 0.01561666838824749
Test accuracy: 0.9944000244140625

result

⑤ How to change the input layer of the model

Basically, fine tuning can be performed by following the steps ➊ to ➍ below.

➊ Download learned weights

With the API of " keras.applications.densenet.DenseNet121 ", you can easily ** download the weights trained by imagenet with Densenet121 **. It's hard to download every time (33MB), so save it in Google Drive.

from keras.applications.densenet import *

model = DenseNet121(weights='imagenet')
model.save(fullpath)

➋ MNIST data conversion

Converts MNIST images from (28, 28, 1) to (64, 64, 1) according to the input format (height, width, channels) of DenseNet121.

import cv2
import numpy as  np
from keras.datasets import mnist

img_w, img_h, img_ch = 64, 64, 1
num_classes = 10

def pretreatment(x):
    #Resize image
    X = []
    for i in range(len(x)):
        dst = cv2.resize(x[i], (img_h, img_w), interpolation=cv2.INTER_CUBIC)
#       dst = cv2.cvtColor(dst, cv2.COLOR_GRAY2RGB)← Do not execute because it is 1ch
        X.append(dst)

    #Reshape
    X = np.array(X).reshape(len(x), img_h, img_w, img_ch)

    #Change data in the range 0 to 255 to the range 0 to 1
    ret = X.astype('float32') / 255
    return ret

#■ Read MNIST data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_valid, y_train, y_valid = train_test_split(x_train, y_train, test_size=0.2)

#■ Learning(Train)data
x_train = pretreatment(x_train)
y_train = np_utils.to_categorical(y_train, num_classes)

#■ Learning(Valid)data
x_valid = pretreatment(x_valid)
y_valid = np_utils.to_categorical(y_valid, num_classes)

#■ Test data
x_test = pretreatment(x_test)
y_test = np_utils.to_categorical(y_test, num_classes)

➌ Build a new network using DenseNet121

(1) Add your own input layer before the input layer of DenseNet121

from keras.layers import *
from keras.models import Model, Sequential

input_tensor = x_train.shape[1:]
input_model = Sequential()
input_model.add(InputLayer(input_shape=input_tensor))
input_model.add(Conv2D(3, (3, 3), padding='same'))
input_model.add(BatchNormalization())
input_model.add(Activation('relu'))

(2) DneseNet121 network construction

With keras, you can easily ** build a network of Densenet121 ** by using the API of " keras.applications.densenet.DenseNet121 ". By setting include_top to False, you can prevent the fully connected layer on the output layer side of the network from being included. Also, do not read weights at this point.

model = DenseNet121(include_top=False, weights=None, input_tensor=input_model.output)

(3) Addition of output layer

DenseNet121 (ImageNet) discriminates 1000 classes as image classification, but since this time we will implement MNIST, we will make it possible to classify into 10 classifications from 0 to 9.

x = Flatten()(model.layers[-1].output)
x = Dense(num_classes, activation="softmax")(x)
return Model(model.inputs, x)

➍ Read learned weights

After configuring the network, load the ** weights downloaded in ➊ **. The weights are loaded by this " load_weights "method.

model.load_weights(fullpath, by_name=True)

➎ Learn

After that, if you learn as usual, you can do ** Fine Tuning **.

➏ Execution result

● TEST: Verification result
Test loss    : 0.019007695838809013
Test accuracy: 0.9934999942779541

result

⑥ Source code

When training this time, ImageDataGenerator is used to train including the following rotated / reduced images, and generalization performance is also taken into consideration.

** Click here for the entire source code ** ▶ github

⑦ and above

Thank you for your hard work. Since it is based on the ** ImageNet trained DenseNet 121 model **, the amount of calculation for 1 epoc is large, but since it is ** Fine Tuning **, it almost converges in about 30 epoc. The correct answer rate and loss value are also reasonably good. This time, we read the weights for ** FineTuning **, but it can be applied to cases such as when learning is performed halfway and learning is started again, so it is good to understand this procedure.

Recommended Posts

[MNIST] I tried Fine Tuning using the ImageNet model.
I tried using the checkio API
I tried refactoring the CNN model of TensorFlow using TF-Slim
I tried the TensorFlow tutorial MNIST 3rd
I tried CNN fine tuning with Resnet
I tried using the BigQuery Storage API
I tried using the trained model VGG16 of the deep learning library Keras
I tried using scrapy for the first time
vprof --I tried using the profiler for Python
I tried using PyCaret at the fastest speed
I tried using the Google Cloud Vision API
I tried using the Datetime module by Python
I tried using the image filter of OpenCV
I tried using the functional programming library toolz
I tried using argparse
I tried using anytree
I tried using aiomysql
I tried using Summpy
I tried using coturn
I tried using "Anvil".
I tried using Hubot
I tried using ESPCN
I tried using PyCaret
I tried using cron
I tried using ngrok
I tried using face_recognition
I tried using Jupyter
I tried using PyCaret
I tried using Heapq
I tried using doctest
I tried using folium
I tried using jinja2
I tried using folium
I tried using time-window
I tried hosting a Pytorch sample model using TorchServe
[Linux] I tried using the genetic statistics software PLINK
I tried clustering ECG data using the K-Shape method
I tried to approximate the sin function using chainer
PyTorch Learning Note 2 (I tried using a pre-trained model)
I tried to identify the language using CNN + Melspectogram
I tried to complement the knowledge graph using OpenKE
I tried to compress the image using machine learning
[I tried using Pythonista 3] Introduction
I tried using easydict (memo).
I tried face recognition using Face ++
I tried using Random Forest
I tried using BigQuery ML
I tried using Amazon Glacier
I tried the changefinder library!
I tried using git inspector
[Python] I tried using OpenPose
I tried using magenta / TensorFlow
I tried using AWS Chalice
I tried using Slack emojinator
I tried using the Python library from Ruby with PyCall
I tried to simulate ad optimization using the bandit algorithm.
I tried face recognition of the laughter problem using Keras.
I tried hosting a TensorFlow deep learning model using TensorFlow Serving
I tried using the DS18B20 temperature sensor with Raspberry Pi
[TF] I tried to visualize the learning result using Tensorboard
Miscellaneous notes that I tried using python for the matter