"Deep Learning from scratch" Self-study memo (No. 19) Data Augmentation

While reading "Deep Learning from scratch" (written by Yasuki Saito, published by O'Reilly Japan), I will make a note of the sites I referred to. Part 18

I've been able to judge dogs and cats reasonably well, but I feel that the correct answer rate is still less than 90%, so I think I'll try the Data Augmentation described on page 245 of the book. ..

Data Augmentation What seems to be easy to do by expanding data Inversion rotation Move Is it around?

So, as I saw it on GradCAM, it seems that the cat responds to the curled back posture and the dog responds to the nose.

That means

Isn't it possible to improve the accuracy of cat identification by adding a rotated or inverted image of a cat? If you enlarge the area around the face of the dog image, the dog's identification accuracy will improve. What can be considered.

Therefore, I would like to verify how adding extended data enhances learning.

First, recreate the training data and test data

The data used so far is made fairly well, just because it is only necessary to confirm the operation of the program. There are only 100 test data items.

I want to save about 1000 test data at random. For the rest of the training data, merge the dog and cat images and then rearrange them randomly. The extended data will be separated for each dog and cat processing method, and will be merged with the training data during learning so that the effect can be verified. In the verification, not only the overall correct answer rate, but also the correct answer rate for dogs and the correct answer rate for cats are verified. In addition, check which features the incorrect image responds to with GradCAM.

With such a policy, we recreated the training data.

def rnd_list(motoarray, toridasi):
  #Create an integer list from 0 to the number of data in the original np array
  #After sorting randomly
  #Returns a list of the specified number of integers and a list of the remaining integers
  import random
  import numpy as np 

  kensuu , tate, yoko, channel = motoarray.shape
  moto = list(range(0, kensuu))
  random.shuffle(moto)
  sel = moto[0:toridasi]
  nokori=moto[toridasi:]
  return sel, nokori

def bunkatu(motoarray, toridasi, lblA):
  #np array data
  #In the specified number list and the rest of the list
  #To divide
  sel, nokori = rnd_list(motoarray, toridasi)

  tsl = []
  tsi = []
  trl = []
  tri = []
  for i in sel:
    imgA = dogimg[i]
    tsl.append(lblA)
    tsi.append(imgA)

  for i in nokori:
    imgA = dogimg[i]
    trl.append(lblA)
    tri.append(imgA)
  return tsl, tsi, trl, tri

def rnd_arry(tri, trl):
  #An array of images and an array of labels
  #Randomly sorted
  #List and return
  sel, nokori = rnd_list(tri, 0)
  wtri = []
  wtrl = []
  for i in nokori:
    imgA = tri[i]
    lblA = trl[i]
    wtri.append(imgA)
    wtrl.append(lblA)
  return wtri, wtrl

#Divide for training and testing and integrate dogs and cats
bunkatusuu = 500
ctsl, ctsi, ctrl, ctri = bunkatu(catimg, bunkatusuu, 0)
dtsl, dtsi, dtrl, dtri = bunkatu(dogimg, bunkatusuu, 1)

tri=np.append(ctri, dtri, axis=0)
trl=np.append(ctrl, dtrl, axis=0)
tsi=np.append(ctsi, dtsi, axis=0)
tsl=np.append(ctsl, dtsl, axis=0)

#Sort randomly
wtri, wtrl = rnd_arry(tri, trl)
wtsi, wtsl = rnd_arry(tsi, tsl)

#save

dataset = {}
dataset['test_label']  = np.array(wtsl, dtype=np.uint8)
dataset['test_img']    = np.array(wtsi, dtype=np.uint8)
dataset['train_label'] = np.array(wtrl, dtype=np.uint8)
dataset['train_img']   = np.array(wtri, dtype=np.uint8) 

import pickle

save_file = '/content/drive/My Drive/Colab Notebooks/deep_learning/dataset/catdog.pkl'    
with open(save_file, 'wb') as f:
    pickle.dump(dataset, f, -1) 

Training data 23994 (dog 11997, cat 11997), test data 1000 (dog 500, cat 500) Is done.

If you input this and process it with DeepConvNet made in Part 18

Epoch 1/10 188/188 [==============================] - 373s 2s/step - loss: 0.7213 - accuracy: 0.5663 Epoch 2/10 188/188 [==============================] - 373s 2s/step - loss: 0.6378 - accuracy: 0.6290 Epoch 3/10 188/188 [==============================] - 373s 2s/step - loss: 0.5898 - accuracy: 0.6713 Epoch 4/10 188/188 [==============================] - 374s 2s/step - loss: 0.5682 - accuracy: 0.6904 Epoch 5/10 188/188 [==============================] - 373s 2s/step - loss: 0.5269 - accuracy: 0.7128 Epoch 6/10 188/188 [==============================] - 374s 2s/step - loss: 0.4972 - accuracy: 0.7300 Epoch 7/10 188/188 [==============================] - 372s 2s/step - loss: 0.4713 - accuracy: 0.7473 Epoch 8/10 188/188 [==============================] - 374s 2s/step - loss: 0.4446 - accuracy: 0.7617 Epoch 9/10 188/188 [==============================] - 373s 2s/step - loss: 0.4318 - accuracy: 0.7665 Epoch 10/10 188/188 [==============================] - 376s 2s/step - loss: 0.4149 - accuracy: 0.7755 32/32 - 4s - loss: 0.3811 - accuracy: 0.8420

The result is a correct answer rate of 84.2%.

I will verify the contents of the result

predictions = model.predict(x_test)

#List the subscripts for misjudgment

gohantei = []
kensuu, w = predictions.shape
for i in range(kensuu):
  predictions_array = predictions[i]
  predicted_label = np.argmax(predictions_array)
  true_label = t_test[i]
  if predicted_label != true_label:
    gohantei.append(i)

print(len(gohantei))

158

There were 158 false positives.

def plot_image(i, predictions, t_label, img):
    class_names = ['cat', 'dog']
    predictions_array = predictions[i]
    img = img[i].reshape((80, 80, 3))
    true_label = t_label[i]
    plt.grid(False)
    plt.xticks([])
    plt.yticks([])

    plt.imshow(img, cmap=plt.cm.binary)

    predicted_label = np.argmax(predictions_array)
    if predicted_label == true_label:
        color = 'blue'
    else:
        color = 'red'

    plt.xlabel("{} {:2.0f}% ({})".format(class_names[predicted_label],
                                    100*np.max(predictions_array),
                                    class_names[true_label]),
                                    color=color)

num_cols = 10
num_rows = int(len(gohantei) / num_cols ) + 1

plt.figure(figsize=(2*num_cols, 2.5*num_rows))
j = 0
for i in gohantei:
  inuneko = t_test[i]
  if inuneko == 0:
    plt.subplot(num_rows, num_cols, j+1)
    plot_image(i, predictions, t_test, x_test)
    j +=1
plt.show()
print("I made a mistake with the cat",j)

plt.figure(figsize=(2*num_cols, 2.5*num_rows))
j = 0
for i in gohantei:
  inuneko = t_test[i]
  if inuneko == 1:
    plt.subplot(num_rows, num_cols, j+1)
    plot_image(i, predictions, t_test, x_test)
    j +=1
plt.show()
print("I made a mistake in the dog",j)

c1.jpg c2.jpg c3.jpg d1.jpg

The breakdown of misjudgment was 109 cats and 49 dogs. Cats are more than twice as misjudged as dogs.

In the first place, is learning enhanced by adding extended data?

Let's see if cat misjudgment can be reduced by expanding and adding cat data.

#Extract only cat images
catdatalist = []
kensuu = len(dataset['train_img'])
for i in range(kensuu):
  label = dataset['train_label'][i]
  if label == 0:
    catdatalist.append(i)
print(len(catdatalist))

11997

#Create a left-right inverted image dataset of a cat
trl = []
tri = []
lbl = 0
for i in catdatalist:
  img = dataset['train_img'][i]
  img = img[:, ::-1, :]
  trl.append(lbl)
  tri.append(img)

catdataset = {}
catdataset['train_label'] = np.array(trl, dtype=np.uint8)
catdataset['train_img']   = np.array(tri, dtype=np.uint8) 

tri =np.append(dataset['train_img'], catdataset['train_img'], axis=0) 
trl =np.append(dataset['train_label'], catdataset['train_label'], axis=0) 
x_train = tri  / 255.0
t_train = trl

Train with the training data with the inverted cat data added.

model.fit(x_train, t_train,  epochs=10, batch_size=128)

Epoch 1/10 282/282 [==============================] - 571s 2s/step - loss: 0.6604 - accuracy: 0.6783 Epoch 2/10 282/282 [==============================] - 569s 2s/step - loss: 0.5840 - accuracy: 0.7220 Epoch 3/10 282/282 [==============================] - 570s 2s/step - loss: 0.5407 - accuracy: 0.7511 Epoch 4/10 282/282 [==============================] - 572s 2s/step - loss: 0.5076 - accuracy: 0.7689 Epoch 5/10 282/282 [==============================] - 565s 2s/step - loss: 0.4808 - accuracy: 0.7860 Epoch 6/10 282/282 [==============================] - 566s 2s/step - loss: 0.4599 - accuracy: 0.7974 Epoch 7/10 282/282 [==============================] - 563s 2s/step - loss: 0.4337 - accuracy: 0.8115 Epoch 8/10 282/282 [==============================] - 565s 2s/step - loss: 0.4137 - accuracy: 0.8181 Epoch 9/10 282/282 [==============================] - 564s 2s/step - loss: 0.3966 - accuracy: 0.8256 Epoch 10/10 282/282 [==============================] - 565s 2s/step - loss: 0.3759 - accuracy: 0.8331

test_loss, test_acc = model.evaluate(x_test,  t_test, verbose=2)

32/32 - 4s - loss: 0.3959 - accuracy: 0.8220

predictions = model.predict(x_test)
#List the subscripts for misjudgment

gohantei = []
kensuu, w = predictions.shape
for i in range(kensuu):
  predictions_array = predictions[i]
  predicted_label = np.argmax(predictions_array)
  true_label = t_test[i]
  if predicted_label != true_label:
    gohantei.append(i)

print(len(gohantei))

178

def plot_image(i, predictions, t_label, img):
    class_names = ['cat', 'dog']
    predictions_array = predictions[i]
    img = img[i].reshape((80, 80, 3))
    true_label = t_label[i]
    plt.grid(False)
    plt.xticks([])
    plt.yticks([])

    plt.imshow(img, cmap=plt.cm.binary)

    predicted_label = np.argmax(predictions_array)
    if predicted_label == true_label:
        color = 'blue'
    else:
        color = 'red'

    plt.xlabel("{} {:2.0f}% ({})".format(class_names[predicted_label],
                                    100*np.max(predictions_array),
                                    class_names[true_label]),
                                    color=color)

num_cols = 10
num_rows = int(len(gohantei) / num_cols ) + 1

plt.figure(figsize=(2*num_cols, 2.5*num_rows))
j = 0
for i in gohantei:
  inuneko = t_test[i]
  if inuneko == 0:
    plt.subplot(num_rows, num_cols, j+1)
    plot_image(i, predictions, t_test, x_test)
    j +=1
plt.show()
print("I made a mistake with the cat",j)

plt.figure(figsize=(2*num_cols, 2.5*num_rows))
j = 0
for i in gohantei:
  inuneko = t_test[i]
  if inuneko == 1:
    plt.subplot(num_rows, num_cols, j+1)
    plot_image(i, predictions, t_test, x_test)
    j +=1
plt.show()
print("I made a mistake in the dog",j)

Wrong cat 28 Wrong dog 150

When the inverted data is not entered There were 158 mistakes, 109 mistakes for cats, and 49 mistakes for dogs. Only for cats, the accuracy is greatly improved. However, the accuracy of the dog decreased by that amount, and the accuracy of the dog as a whole also decreased.

Learning seems to be strengthened by adding data, but does it mean that the side effects are also large?

I examined the image of a cat that changed from a mistake to a correct answer and the image of a dog that changed from a correct answer to a mistake.

The cat image that changed from a mistake to the correct answer The red mark is the one that was wrong the second time mae1.jpg mae2.jpg mae3.jpg

The dog image that changed from the correct answer to the wrong answer The yellow mark was the first mistake matigai2.jpg matigai3.jpg matigai4.jpg matigai5.jpg

The image of a cat sitting like a dog is now correctly judged, but the image of a dog sitting normally is also judged as a "cat"? Also, in the first time, a large copy of the face, which would have been judged as a "dog" only by the nose, is judged as a "cat".

Woom. I don't know.

So what happens if you learn by adding an inverted image of the dog? I tried it.

I tried to learn by adding inverted images of both cats and dogs

In other words, the number of training data will be doubled.

Result is

32/32 - 4s - loss: 0.2186 - accuracy: 0.9090

Corrected answer rate improved to 90%. Of the 91 misjudgments, 48 were mistaken for cats as dogs and 43 were mistaken for dogs as cats.

When the inverted data is not entered There were 158 mistakes, 109 mistakes for cats, and 49 mistakes for dogs. The number of mistaken cats has been halved.

It seems that the image of sitting like a dog and the image of a large face are changed from mistakes to correct answers.

From the above, can we say the following?

-The training data is flipped horizontally, and even if the number of cases is doubled, it can be used for learning and is effective. ・ When classifying into two as in this example, it is better to have the same number of training data for dogs and cats for more unbiased learning.

However, if the number of training data doubles, Google Colab will also run out of RAM and crash. So, it seems difficult to increase the data any more, so we will end the discrimination of dog and cat data at this point.

Part 18

Recommended Posts

"Deep Learning from scratch" Self-study memo (No. 19) Data Augmentation
"Deep Learning from scratch" self-study memo (No. 19-2) Data Augmentation continued
"Deep Learning from scratch" Self-study memo (No. 11) CNN
"Deep Learning from scratch 2" Self-study memo (No. 21) Chapters 3 and 4
"Deep Learning from scratch" Self-study memo (Part 12) Deep learning
"Deep Learning from scratch" self-study memo (No. 18) One! Meow! Grad-CAM!
"Deep Learning from scratch" self-study memo (No. 15) TensorFlow beginner tutorial
"Deep Learning from scratch" self-study memo (unreadable glossary)
"Deep Learning from scratch" Self-study memo (10) MultiLayerNet class
"Deep Learning from scratch" self-study memo (No. 13) Try using Google Colaboratory
"Deep Learning from scratch" Self-study memo (No. 10-2) Initial value of weight
[Learning memo] Deep Learning made from scratch [Chapter 7]
Deep learning / Deep learning made from scratch Chapter 6 Memo
[Learning memo] Deep Learning made from scratch [Chapter 5]
[Learning memo] Deep Learning made from scratch [Chapter 6]
Deep learning / Deep learning made from scratch Chapter 7 Memo
[Learning memo] Deep Learning made from scratch [~ Chapter 4]
Deep Learning from scratch
"Deep Learning from scratch" Self-study memo (No. 16) I tried to build SimpleConvNet with Keras
"Deep Learning from scratch" Self-study memo (No. 17) I tried to build DeepConvNet with Keras
Deep Learning from scratch Chapter 2 Perceptron (reading memo)
[Learning memo] Deep Learning from scratch ~ Implementation of Dropout ~
"Deep Learning from scratch" Self-study memo (No. 14) Run the program in Chapter 4 on Google Colaboratory
Deep Learning from scratch 1-3 chapters
Deep Learning / Deep Learning from Zero 2 Chapter 4 Memo
Deep Learning / Deep Learning from Zero Chapter 3 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 5 Memo
Deep learning from scratch (cost calculation)
Deep Learning / Deep Learning from Zero 2 Chapter 7 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 8 Memo
Deep Learning / Deep Learning from Zero Chapter 5 Memo
Deep Learning / Deep Learning from Zero Chapter 4 Memo
Deep Learning / Deep Learning from Zero 2 Chapter 3 Memo
Deep Learning memos made from scratch
Deep Learning / Deep Learning from Zero 2 Chapter 6 Memo
Deep learning from scratch (forward propagation edition)
Deep learning / Deep learning from scratch 2-Try moving GRU
"Deep Learning from scratch" in Haskell (unfinished)
Why ModuleNotFoundError: No module named'dataset.mnist' appears in "Deep Learning from scratch".
[Windows 10] "Deep Learning from scratch" environment construction
Learning record of reading "Deep Learning from scratch"
"Deep Learning from scratch" Self-study memo (Part 8) I drew the graph in Chapter 6 with matplotlib
[Deep Learning from scratch] About hyperparameter optimization
Deep Learning from scratch ① Chapter 6 "Techniques related to learning"
Good book "Deep Learning from scratch" on GitHub
Python vs Ruby "Deep Learning from scratch" Summary
Django memo # 1 from scratch
[Deep Learning from scratch] I implemented the Affine layer
Application of Deep Learning 2 made from scratch Spam filter
[Deep Learning from scratch] I tried to explain Dropout
Deep learning / LSTM scratch code
[Deep Learning from scratch] Implementation of Momentum method and AdaGrad method
An amateur stumbled in Deep Learning from scratch Note: Chapter 1
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 5
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 2
Create an environment for "Deep Learning from scratch" with Docker
An amateur stumbled in Deep Learning from scratch Note: Chapter 3
An amateur stumbled in Deep Learning from scratch Note: Chapter 7
An amateur stumbled in Deep Learning from scratch Note: Chapter 5
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 7
Making from scratch Deep Learning ❷ An amateur stumbled Note: Chapter 1