How to use Keras ~ From simple model generation to CNN ~

What to do in this article

In deep learning, the implementation procedure differs depending on the backend used. Therefore, you will learn how to use it by reading the official documents and referring to the reference books that explain it. This article explains the flow of how to use it.

Implementation procedure when using keras

1. Define training data

2. Define a multi-layer network that maps input values to target values.

3. Set up the learning process by selecting the loss function, optimizer, and indicators to monitor.

4. Train the training data repeatedly by calling the fit method of the model.

Build keras python environment

Create a new virtual environment on anaconda and install tensorflow and keras. From now on, it will be executed on the created environment. Do the following on the anaconda prompt:

conda create -n keras_env python=3.6 #Creating a virtual environment
conda activate keras_env #Switching environment
conda install tensorflow==1.12.0
conda isntall keras==2.2.4

Specific procedure

Data set loading

from keras.datasets import imdb

(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)

Data preprocessing

Data vectorization

All neural network input and objective values must be ** tensors of floating point data. ** No matter what data you have to process, such as audio, images, text, you first need to convert them to a tensor. To vectorize the classification labels, use *** one-hot encoding ***.

from keras.utils.np_utils import to_categorical

one_hot_train_labels = to_categorical(train_labels)
one_hot_test_labels = to_categorical(test_labels)

Data normalization

The image data is encoded as an integer representing a grayscale value in the range 0-255. To supply this data to the neural network, cast it with float32 type and then divide it by 255 to convert it to a floating point number in the range 0 to 1. Supplying features with different ranges to the network is a problem at all costs. Therefore, normalization is performed to make the range the same. The process of subtracting the average value of the features and dividing by the standard deviation is performed. Then, the center of the feature becomes 0 and the standard deviation becomes 1.

Model definition

Set the activation function and the number of neurons for each layer. Add them in the order you want them to propagate with ʻadd ()`.

from keras import models
from keras import layers

model = models.Sequential()
model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))
model.add(layers.Dense(16, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.add(Dense(200))
model.add(Activation("relu"))

Compiling the model

** Select the optimizer and loss function. ** In the following cases, it is specified as a character string, but this is possible because it is packaged as part of keras.

model.compile(optimizer='rmsprop',#Character specification
              loss='binary_crossentropy',
              metrics=['accuracy'])

** If you want to specify the parameter argument of the optimizer **, specify the instance of the optimizer class and call the method as follows.

from keras import optimizers

model.compile(optimizer=optimizers.RMSprop(lr=0.001),#Method specification
              loss='binary_crossentropy',
              metrics=['accuracy'])

** If you want to use your own loss function or indicator function **, specify a function object as an argument to the loss parameter or the metrics parameter.

from keras import losses
from keras import metrics

model.compile(optimizer=optimizers.RMSprop(lr=0.001),
              loss=losses.binary_crossentropy,
              metrics=[metrics.binary_accuracy])

Setting a validating data set

To monitor the accuracy rate when training a model with entirely new data, create a validation dataset using a sample set aside from the original training dataset. In the following, when taking out 10000 samples.

x_val = x_train[:10000] #Extraction of verification data
partial_x_train = x_train[10000:] #

y_val = y_train[:10000] #Extraction of correct answer verification data
partial_y_train = y_train[10000:]

k-validated cross-validation

If the number of data is small, the verification data will be quite small. As a result, the validation score may vary significantly depending on which data points are selected for validation and training. That is, depending on the method of dividing the verification data set, the variance of the verification score becomes high, resulting in overfitting. The best way to prevent this is ** k-validation **. I won't explain it in detail, so please check it out.

Model training

Train 20 epochs in 8 mini-batch. x is training data y is often correct data. Monitor the loss value and correct answer rate for the 10000 samples set aside. Validation data is passed as an argument to the validation_data parameter.

history = model.fit(partial_x_train,
                    partial_y_train,
                    epochs=20,
                    batch_size=8,
                    validation_data=(x_val, y_val))

The fit method returns the output trained for each epoch and the output of verification data in dictionary type. This time, it is saved in the history object. When you run the following code

history_dict = history.history
history_dict.keys()

dict_keys(['val_acc', 'acc', 'val_loss', 'loss']) Will be.

Plot loss values in training and validation data

Plot the loss value using matplotlib. Since the training is recorded in the history object, call it from here. Adjust hyperparameters based on this result.

import matplotlib.pyplot as plt

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(1, len(acc) + 1)

# "bo" is for "blue dot"
plt.plot(epochs, loss, 'bo', label='Training loss')
# b is for "solid blue line"
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()

plt.show()

Procedure for image recognition on CNN

Copy of image

Create a folder to save the training image (train) and a folder to save the validation image (validation), and copy and sort the collected images for use in learning. The number of images in the verification data is determined by the number of images, but it is recommended to adjust it to about 20% to 40% of the total number of images. Specify the path of the folder created here in the folder path of flow_from_directory () that appears in the later preprocessing process.

CNN instantiation

from keras import layers
from keras import models

model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(Dense(2))
model.add(Activation("softmax"))

The shape of the input tensor of CNN is (image_height, image_width, image_channels). The third argument is the number of image channels. In the case of RGB images, the number of channels is 3. Conv2D (depth of output feature, filter size) If the argument of padding ='same' is specified, the width and height of the output are padded so that they are the same as the input. By the way, you can check the model created by model.summary (). In the case of image classification, the final layer is Dense (fully connected layer), and the number of classes to be classified is specified in the argument. Since the output is the probability of judgment, select softmax as the loss function.

Compiling the model

Loss function and optimizer settings.

from keras import optimizers

model.compile(loss='binary_crossentropy',
              optimizer=optimizers.RMSprop(lr=1e-4),
              metrics=['acc'])

Data preprocessing

Before you can supply data to CNN, it must be properly processed as a floating-point tensor. The procedure is as follows,

1. 1. Load image file
2. Decode the contents of a JPEG file into an RGB pixel grid.
3. 3. Convert these pixels to a floating point tensor.
4. Rescale to pixels (0 to 255) and set values in the range [0,1]. keras has a utility that automatically handles the above steps. You can use the ʻImageDataGenerator` class to set up a python generator that can automatically convert image files on disk to a batch of preprocessed tensors.

Load images from a directory using ImageDataGenerator

from keras.preprocessing.image import ImageDataGenerator

# All images will be rescaled by 1./255
train_datagen = ImageDataGenerator(rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
        # This is the target directory
        train_dir,
        # All images will be resized to 150x150
        target_size=(150, 150),
        batch_size=20,
        # Since we use binary_crossentropy loss, we need binary labels
        class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
        validation_dir,
        target_size=(150, 150),
        batch_size=20,
        class_mode='binary')

Model training

Use the fit_generator () function to train the model. steps_per_epoch is the number of steps to proceed in one epoch. validation_steps indicates how many images are validated in one epoch.

history = model.fit_generator(
      train_generator,
      steps_per_epoch=10,
      epochs=30,
      validation_data=validation_generator,
      validation_steps=5)

Network storage

Use the save () function to save the parameters of the trained network. In keras, it is saved as a file with the extension .h5. If you call the h5 file, you can predict the image with this parameter from the next time onward.

model.save('cats_and_dogs_small_1.h5')

Correct answer rate plot

import matplotlib.pyplot as plt

acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs = range(len(acc))

plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()

plt.figure()

plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()

plt.show()

Estimate on the saved network

Just change the part that was set to model = models.Sequential () as follows. Specify the path of the saved network file in the argument of load_model. Since the trained network is used, the ʻadd ()` system that was used at the time of learning is unnecessary. The trained model is loaded as it is.

model=keras.models.load_model('cats_and_dogs_small_1.h5')

reference

https://qiita.com/GushiSnow/items/8c946208de0d6a4e31e7#%E5%85%B7%E4%BD%93%E7%9A%84%E3%81%AA%E5%AE%9F%E8%A3%85%E4%BE%8B https://qiita.com/tsunaki/items/608ff3cd941d82cd656b https://qiita.com/tomo_20180402/items/e8c55bdca648f4877188 https://ymgsapo.com/2019/02/28/keras-dog-and-cat/