PyTorch learning memo (I made the same model as Karas)

Introduction

I had thought that I wouldn't study PyTorch soon, but I had a lot of time, so I studied with a book (PyTorch Neural Network Implementation Handbook) in one hand. The prerequisite knowledge of the author at the time of writing this article is

is. I think there are other people who are similar, so I hope you find it helpful.

Rough impression of PyTorch

After all, it's only half a day since I started studying, so it may be wrong, but as a rough impression, I thought that the implementation complexity was just halfway between Keras and TensorFlow. The definition of the model is almost the same as Keras. On the contrary, the learning part gave me the impression that I had to write a fair amount of bare code. However, it wasn't that hard because it worked just by copying the code that was working elsewhere. On the flip side, Keras, which can hide this area cleanly, may be wonderful.

Method of approach

I don't use my head to copy the code of the book, so I decided to create a model with exactly the same structure and algorithm as the one I implemented in Keras for the classification of CIFAR-10 that I had tried in the past. .. There were some things I noticed for the first time when I implemented it from scratch, so I think it was an efficient study method.

Implementation code and environment

The implementation code explained below is uploaded on Github below. As anyone who has done it will know, CIFAR-10 is a fairly difficult example, and if you make a model normally, the accuracy is only about 60%. I think that the model introduced here is certainly the structure of the highly accurate model that appeared in Kaggle. (Kaggle also did other data pre-processing, but that's omitted). The accuracy is different every time, but if it is in good condition, it will go up to about 80%.

https://github.com/makaishi2/sample-data/blob/master/notebooks/cifar10_keras.ipynb

https://github.com/makaishi2/sample-data/blob/master/notebooks/cifar10_pytorch.ipynb

I used Google Colab as the environment. The Keras / PyTorch relationship did not require any additional installation. I think that only the Japanese localization module of matplotlib was additionally introduced. As expected, Google Colab is very convenient in such cases.

Code commentary

With this as a preface, let's get into the actual code description.

Data loading

As for the training data of CIFAR-10, like Keras, PyTorch has a function and I could load it immediately. Since it's a big deal, I'll compare the code for this part as well.

Keras

#Import Keras library

from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense, Activation

#Import of other libraries

!pip install japanize_matplotlib | tail -n 1
import matplotlib.pyplot as plt
import japanize_matplotlib
import numpy as np

#Read training data

from keras.datasets import cifar10
(x_train, y_train), (x_test, y_test) = cifar10.load_data()

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

#Calculation of the number of classes to be classified
class_labels_count = len(set(y_train.flatten()))

# One Hot Encoding
from keras.utils import np_utils
y_train_ohe = np_utils.to_categorical(y_train, class_labels_count)
y_test_ohe = np_utils.to_categorical(y_test, class_labels_count)

PyTorch

#PyTooch related library import

import torch
import torchvision
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torchvision.transforms as transforms

#Other library import

%matplotlib inline
import numpy as np
!pip install japanize_matplotlib | tail -n 1
import matplotlib.pyplot as plt
import japanize_matplotlib

#Number of classification classes
num_classes = 10

#Number of learning repetitions
nb_epoch = 20

#How many images to use in one learning
batch_size = 128

#Read training data

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
train_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,download=True, transform=transform)
test_loader = torch.utils.data.DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

PyTorch was different from Keras in terms of how it handles training data. Classes specialized for learning called DataSet and DataLoader have been created, and they will be used. A DataSet is a tuple of input data and a set of correct label values, and is prepared as an Iterator. The input data is a PyTorch-specific variable of the class Tensor. The specification of batch_size used in the so-called "** mini-batch learning method **" is done in the DataLoader (I didn't notice this at all at first). Another point is that there is a difference in the correct label data. In the case of Keras, the correct answer data must be One Hot Encoding. We implement it at the end of the code. In PyTorch, it seems that the encoding of the label value is done inside the framework, and the value without encoding (value such as 6 or 3) can be used as it is as the correct answer value at the time of learning. PyTorch seems to be more convenient in this regard.

Image display of input data

It has nothing to do with the learning itself, but since it's a big deal, let's display the first 10 of the read learning data as an image. The result is as follows.

スクリーンショット 2021-01-07 16.05.01.png

Even such a simple thing can be implemented quite differently with Keras / PyTorch due to the difference in how to hold the data.

Keras

plt.figure(figsize=(15, 4))
for i in range(10):
    ax = plt.subplot(1, 10, i + 1)
    image = x_train[i]
    label = y_train[i][0]
    plt.imshow(image)
    ax.set_title(classes[label], fontsize=16)
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()

PyTorch

plt.figure(figsize=(15, 4))
for i in range(10):
    ax = plt.subplot(1, 10, i + 1)
    image, label = trainset[i]
    np_image = image.numpy().copy()
    img = np.transpose(np_image, (1, 2, 0))
    img2 = (img + 1)/2 
    plt.imshow(img2)
    ax.set_title(classes[label], fontsize=16)
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()

The first feature of PyTorch is that it loops through a variable called trainset and extracts a set of (image, label) one by one. Another difference is that it requires a lot of processing to display the image data on the screen. In the case of Keras, the data was originally a numpy array, and it could be displayed by passing it to the plt.imshow function as it is, but in the case of PyTorch, the following three steps of processing were required.

I didn't notice the last story, but when I understood it, it seems that the data is obtained by processing with the transforms.Normalize function used when reading the data. Well, if you have the state before processing with another variable, you should be able to implement it more efficiently, but since I was a beginner of PyTorch and did not know what to do specifically, I am OK with this once.

Model generation

The next step is to generate the most essential model. Regarding this, I have the impression that Keras / PyTorch is almost the same.

Keras

def cnn_model(x_train, class_labels_count):
    model = Sequential()
    model.add(Conv2D(32, (3, 3), padding="same", input_shape=x_train.shape[1:]))
    model.add(Activation('relu'))
    model.add(Conv2D(32, (3, 3))) 
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))
    model.add(Conv2D(64, (3, 3), padding='same'))
    model.add(Activation('relu'))
    model.add(Conv2D(64, (3, 3)))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))
    model.add(Flatten())
    model.add(Dense(512))
    model.add(Activation('relu'))
    model.add(Dropout(0.5))
    model.add(Dense(class_labels_count))
    model.add(Activation('softmax'))

    model.compile(
        loss='categorical_crossentropy',
        optimizer='adam',
        metrics=['accuracy']
    )
    return model

#Model generation

model = cnn_model(x_train, class_labels_count)

PyTorch

#Model class definition

class cifar10_cnn(nn.Module):
  def __init__(self, num_classes):
    super(cifar10_cnn,self).__init__()
    self.conv1 = nn.Conv2d(3, 32, 3, padding=(1,1), padding_mode='replicate')
    self.conv2 = nn.Conv2d(32, 32, 3)
    self.conv3 = nn.Conv2d(32, 64, 3, padding=(1,1), padding_mode='replicate')
    self.conv4 = nn.Conv2d(64, 64, 3)
    self.relu = nn.ReLU(inplace=True)
    self.dropout1 = nn.Dropout(0.25)
    self.dropout2 = nn.Dropout(0.5)
    self.maxpool = nn.MaxPool2d((2,2))
    self.classifier1 = nn.Linear(2304, 512)
    self.classifier2 = nn.Linear(512, num_classes)

    self.features = nn.Sequential(
        self.conv1,
        self.relu,
        self.conv2,
        self.relu,
        self.maxpool,
        self.dropout1,
        self.conv3,
        self.relu,
        self.conv4,
        self.relu,
        self.dropout1,
        self.maxpool)
    
    self.classifier = nn.Sequential(
       self.classifier1,
       self.relu,
       self.dropout2,
       self.classifier2)

  def forward(self, x):
    x = self.features(x)
    x = x.view(x.size(0), -1)
    x = self.classifier(x)
    return x

#Check GPU

device = 'cuda' if torch.cuda.is_available() else 'cpu'
print(device)

#Model instance generation and GPU allocation

net = cifar10_cnn(num_classes).to(device)

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(net.parameters())
      

In Keras, you can define a model with a function (you can do it solidly without using a function), but in PyTorch, it seems to be a good idea to define a class. However, I get the impression that they are similar in terms of the amount of implementation code. There is one point that I had a hard time with PyTorch, and it seems that I can not simply specify padding ='same' like Keras regarding padding. When I googled, I found the following article in the article on qiita, so I used it.

https://qiita.com/syoyo/items/ddff3268b4dfa3ebb3d6

Also, since PyTorch can dynamically declare the structure of the model, I think that the magic number 2304 of nn.Linear (2304, 512) can be calculated, but prioritize moving it first. I proceeded.

In PyTorch, if you want to use GPU, it seems that you need to explicitly assign the device like net = cifar10_cnn (num_classes) .to (device). In Keras, if you have a GPU, you can use it without permission, so Keras was more convenient in that respect.

Model structure confirmation

Let's check if this really has the same structure. Here is the command and its output to check the structure with Keras / PyTorch.

Keras

#Model summary view

model.summary()
result
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 32, 32, 32)        896       
_________________________________________________________________
activation (Activation)      (None, 32, 32, 32)        0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 30, 30, 32)        9248      
_________________________________________________________________
activation_1 (Activation)    (None, 30, 30, 32)        0         
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 15, 15, 32)        0         
_________________________________________________________________
dropout (Dropout)            (None, 15, 15, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 15, 15, 64)        18496     
_________________________________________________________________
activation_2 (Activation)    (None, 15, 15, 64)        0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 13, 13, 64)        36928     
_________________________________________________________________
activation_3 (Activation)    (None, 13, 13, 64)        0         
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 6, 6, 64)          0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 6, 6, 64)          0         
_________________________________________________________________
flatten (Flatten)            (None, 2304)              0         
_________________________________________________________________
dense (Dense)                (None, 512)               1180160   
_________________________________________________________________
activation_4 (Activation)    (None, 512)               0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 10)                5130      
_________________________________________________________________
activation_5 (Activation)    (None, 10)                0         
=================================================================
Total params: 1,250,858
Trainable params: 1,250,858
Non-trainable params: 0
_________________________________________________________________

PyTorch

#Model summary view

from torchsummary import summary
summary(net,(3,32,32))
result
----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1           [-1, 32, 32, 32]             896
            Conv2d-2           [-1, 32, 32, 32]             896
              ReLU-3           [-1, 32, 32, 32]               0
              ReLU-4           [-1, 32, 32, 32]               0
              ReLU-5           [-1, 32, 32, 32]               0
            Conv2d-6           [-1, 32, 30, 30]           9,248
            Conv2d-7           [-1, 32, 30, 30]           9,248
              ReLU-8           [-1, 32, 30, 30]               0
              ReLU-9           [-1, 32, 30, 30]               0
             ReLU-10           [-1, 32, 30, 30]               0
        MaxPool2d-11           [-1, 32, 15, 15]               0
        MaxPool2d-12           [-1, 32, 15, 15]               0
          Dropout-13           [-1, 32, 15, 15]               0
          Dropout-14           [-1, 32, 15, 15]               0
           Conv2d-15           [-1, 64, 15, 15]          18,496
           Conv2d-16           [-1, 64, 15, 15]          18,496
             ReLU-17           [-1, 64, 15, 15]               0
             ReLU-18           [-1, 64, 15, 15]               0
             ReLU-19           [-1, 64, 15, 15]               0
           Conv2d-20           [-1, 64, 13, 13]          36,928
           Conv2d-21           [-1, 64, 13, 13]          36,928
             ReLU-22           [-1, 64, 13, 13]               0
             ReLU-23           [-1, 64, 13, 13]               0
             ReLU-24           [-1, 64, 13, 13]               0
          Dropout-25           [-1, 64, 13, 13]               0
          Dropout-26           [-1, 64, 13, 13]               0
        MaxPool2d-27             [-1, 64, 6, 6]               0
        MaxPool2d-28             [-1, 64, 6, 6]               0
           Linear-29                  [-1, 512]       1,180,160
           Linear-30                  [-1, 512]       1,180,160
             ReLU-31                  [-1, 512]               0
             ReLU-32                  [-1, 512]               0
             ReLU-33                  [-1, 512]               0
          Dropout-34                  [-1, 512]               0
          Dropout-35                  [-1, 512]               0
           Linear-36                   [-1, 10]           5,130
           Linear-37                   [-1, 10]           5,130
================================================================
Total params: 2,501,716
Trainable params: 2,501,716
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.01
Forward/backward pass size (MB): 3.76
Params size (MB): 9.54
Estimated Total Size (MB): 13.31
----------------------------------------------------------------

The number of elements fits perfectly in all layers and looks nice. In the case of PyTorch, if you execute the variable net which is an instance of the model as it is, the following information will also be displayed.

net
cifar10_cnn(
  (conv1): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
  (conv2): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1))
  (conv3): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
  (conv4): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1))
  (relu): ReLU(inplace=True)
  (dropout1): Dropout(p=0.25, inplace=False)
  (dropout2): Dropout(p=0.5, inplace=False)
  (maxpool): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
  (classifier1): Linear(in_features=2304, out_features=512, bias=True)
  (classifier2): Linear(in_features=512, out_features=10, bias=True)
  (features): Sequential(
    (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
    (1): ReLU(inplace=True)
    (2): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1))
    (3): ReLU(inplace=True)
    (4): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
    (5): Dropout(p=0.25, inplace=False)
    (6): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), padding_mode=replicate)
    (7): ReLU(inplace=True)
    (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1))
    (9): ReLU(inplace=True)
    (10): Dropout(p=0.25, inplace=False)
    (11): MaxPool2d(kernel_size=(2, 2), stride=(2, 2), padding=0, dilation=1, ceil_mode=False)
  )
  (classifier): Sequential(
    (0): Linear(in_features=2304, out_features=512, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=512, out_features=10, bias=True)
  )
)


Learning

Now everything is ready. We will finally carry out learning. As I mentioned at the beginning, this part is (currently) overwhelmingly easy for Keras.

Keras

#Number of learning repetitions
nb_epoch = 20

#How many images to use in one learning
batch_size = 128

#Learning

history = model.fit(
    x_train, y_train_ohe, batch_size=batch_size, epochs=nb_epoch, verbose=1,
    validation_data=(x_test, y_test_ohe), shuffle=True
)

PyTorch

Learning

train_loss_list = []
train_acc_list = []
val_loss_list = []
val_acc_list = []

for epoch in range(nb_epoch):
    train_loss = 0
    train_acc = 0
    val_loss = 0
    val_acc = 0
    
    #train
    net.train()
    for i, (images, labels) in enumerate(train_loader):
      #view()Do not convert with
      images, labels = images.to(device), labels.to(device)
      
      optimizer.zero_grad()
      outputs = net(images)
      loss = criterion(outputs, labels)
      train_loss += loss.item()
      train_acc += (outputs.max(1)[1] == labels).sum().item()
      loss.backward()
      optimizer.step()
    
    avg_train_loss = train_loss / len(train_loader.dataset)
    avg_train_acc = train_acc / len(train_loader.dataset)
    
    #val
    net.eval()
    with torch.no_grad():
      for images, labels in test_loader:
        #view()Do not convert with
        images = images.to(device)
        labels = labels.to(device)
        outputs = net(images)
        loss = criterion(outputs, labels)
        val_loss += loss.item()
        val_acc += (outputs.max(1)[1] == labels).sum().item()
    avg_val_loss = val_loss / len(test_loader.dataset)
    avg_val_acc = val_acc / len(test_loader.dataset)
    
    print ('Epoch [{}/{}], loss: {loss:.4f} val_loss: {val_loss:.4f}, val_acc: {val_acc:.4f}' 
                   .format(epoch+1, nb_epoch, i+1, loss=avg_train_loss, val_loss=avg_val_loss, val_acc=avg_val_acc))
    train_loss_list.append(avg_train_loss)
    train_acc_list.append(avg_train_acc)
    val_loss_list.append(avg_val_loss)
    val_acc_list.append(avg_val_acc)

The execution result of each is as follows. However, Keras is the output of the fit function prepared by the framework itself, while PyTorch only outputs the output similar to it by hand-set the print function, so it is not correct to compare in the first place. I can say.

Keras

Epoch 1/20
391/391 [==============================] - 7s 11ms/step - loss: 4.8051 - accuracy: 0.2355 - val_loss: 1.5342 - val_accuracy: 0.4480
Epoch 2/20
391/391 [==============================] - 4s 10ms/step - loss: 1.4853 - accuracy: 0.4612 - val_loss: 1.2996 - val_accuracy: 0.5420
Epoch 3/20
391/391 [==============================] - 4s 10ms/step - loss: 1.3215 - accuracy: 0.5309 - val_loss: 1.1627 - val_accuracy: 0.5996
Epoch 4/20
391/391 [==============================] - 4s 10ms/step - loss: 1.2014 - accuracy: 0.5732 - val_loss: 1.0446 - val_accuracy: 0.6388
Epoch 5/20
391/391 [==============================] - 4s 10ms/step - loss: 1.1124 - accuracy: 0.6070 - val_loss: 0.9813 - val_accuracy: 0.6627
Epoch 6/20
391/391 [==============================] - 4s 10ms/step - loss: 1.0317 - accuracy: 0.6355 - val_loss: 0.9245 - val_accuracy: 0.6772
Epoch 7/20
391/391 [==============================] - 4s 10ms/step - loss: 0.9625 - accuracy: 0.6639 - val_loss: 0.8732 - val_accuracy: 0.7022
Epoch 8/20
391/391 [==============================] - 4s 10ms/step - loss: 0.9309 - accuracy: 0.6748 - val_loss: 0.8433 - val_accuracy: 0.7064
Epoch 9/20
391/391 [==============================] - 4s 10ms/step - loss: 0.8730 - accuracy: 0.6922 - val_loss: 0.8197 - val_accuracy: 0.7174
Epoch 10/20
391/391 [==============================] - 4s 10ms/step - loss: 0.8575 - accuracy: 0.6982 - val_loss: 0.7722 - val_accuracy: 0.7310
Epoch 11/20
391/391 [==============================] - 4s 10ms/step - loss: 0.8030 - accuracy: 0.7178 - val_loss: 0.7756 - val_accuracy: 0.7272
Epoch 12/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7841 - accuracy: 0.7244 - val_loss: 0.7372 - val_accuracy: 0.7468
Epoch 13/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7384 - accuracy: 0.7429 - val_loss: 0.7738 - val_accuracy: 0.7340
Epoch 14/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7321 - accuracy: 0.7462 - val_loss: 0.7177 - val_accuracy: 0.7501
Epoch 15/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6976 - accuracy: 0.7530 - val_loss: 0.7478 - val_accuracy: 0.7477
Epoch 16/20
391/391 [==============================] - 4s 10ms/step - loss: 0.7047 - accuracy: 0.7537 - val_loss: 0.7160 - val_accuracy: 0.7608
Epoch 17/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6638 - accuracy: 0.7682 - val_loss: 0.7111 - val_accuracy: 0.7630
Epoch 18/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6509 - accuracy: 0.7709 - val_loss: 0.7099 - val_accuracy: 0.7610
Epoch 19/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6346 - accuracy: 0.7770 - val_loss: 0.6933 - val_accuracy: 0.7691
Epoch 20/20
391/391 [==============================] - 4s 10ms/step - loss: 0.6119 - accuracy: 0.7833 - val_loss: 0.7006 - val_accuracy: 0.7619

PyTorch

Epoch [1/20], loss: 0.0124 val_loss: 0.0104, val_acc: 0.5187
Epoch [2/20], loss: 0.0094 val_loss: 0.0081, val_acc: 0.6432
Epoch [3/20], loss: 0.0079 val_loss: 0.0075, val_acc: 0.6791
Epoch [4/20], loss: 0.0069 val_loss: 0.0066, val_acc: 0.7173
Epoch [5/20], loss: 0.0062 val_loss: 0.0060, val_acc: 0.7405
Epoch [6/20], loss: 0.0058 val_loss: 0.0057, val_acc: 0.7557
Epoch [7/20], loss: 0.0053 val_loss: 0.0054, val_acc: 0.7666
Epoch [8/20], loss: 0.0050 val_loss: 0.0054, val_acc: 0.7656
Epoch [9/20], loss: 0.0047 val_loss: 0.0052, val_acc: 0.7740
Epoch [10/20], loss: 0.0045 val_loss: 0.0052, val_acc: 0.7708
Epoch [11/20], loss: 0.0042 val_loss: 0.0051, val_acc: 0.7790
Epoch [12/20], loss: 0.0040 val_loss: 0.0051, val_acc: 0.7797
Epoch [13/20], loss: 0.0038 val_loss: 0.0051, val_acc: 0.7802
Epoch [14/20], loss: 0.0037 val_loss: 0.0050, val_acc: 0.7812
Epoch [15/20], loss: 0.0035 val_loss: 0.0049, val_acc: 0.7910
Epoch [16/20], loss: 0.0034 val_loss: 0.0049, val_acc: 0.7840
Epoch [17/20], loss: 0.0033 val_loss: 0.0049, val_acc: 0.7936
Epoch [18/20], loss: 0.0031 val_loss: 0.0049, val_acc: 0.7917
Epoch [19/20], loss: 0.0030 val_loss: 0.0048, val_acc: 0.8008
Epoch [20/20], loss: 0.0029 val_loss: 0.0050, val_acc: 0.7900

Learning curve

Finally, let's take a look at the learning curve for each case. As you know, in deep learning, the accuracy is different each time because it uses random numbers. Please note that it is not possible to argue which relaim work is better based on this result alone.

Keras

#Learning curve(Loss function value)
plt.figure(figsize=(8,6))
plt.plot(history.history['val_loss'],label='adam', lw=3, c='b')
plt.title('Learning curve(Loss function value)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()

スクリーンショット 2021-01-07 17.06.16.png

#Learning curve(accuracy)
plt.figure(figsize=(8,6))
plt.plot(history.history['val_accuracy'],label='adam', lw=3, c='b')
plt.title('Learning curve(accuracy)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()

スクリーンショット 2021-01-07 17.07.18.png

PyTorch

#Learning curve(Loss function value)
plt.figure(figsize=(8,6))
plt.plot(val_loss_list,label='adam', lw=3, c='b')
plt.title('Learning curve(Loss function value)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()

スクリーンショット 2021-01-07 17.08.51.png

#Learning curve(accuracy)
plt.figure(figsize=(8,6))
plt.plot(val_acc_list,label='adam', lw=3, c='b')
plt.title('Learning curve(accuracy)')
plt.xticks(size=14)
plt.yticks(size=14)
plt.grid(lw=2)
plt.legend(fontsize=14)
plt.show()

スクリーンショット 2021-01-07 17.09.34.png

References

PyTorch Neural Network Implementation Handbook Keiichiro Miyamoto (Author), Yohei Okawa (Author), Takuya Mouri (Author) Shuwa System

Recommended Posts

PyTorch learning memo (I made the same model as Karas)
PyTorch Learning Note 2 (I tried using a pre-trained model)
I made a function to check the model of DCGAN
I made an API with Docker that returns the predicted value of the machine learning model
I made a VGG16 model using TensorFlow (on the way)
I made Word2Vec with Pytorch
I want to manually assign the training parameters of the [Pytorch] model
Validate the learning model with Pylearn2
I tried calling the prediction API of the machine learning model from WordPress
I tried using the trained model VGG16 of the deep learning library Keras
I made a GAN with Keras, so I made a video of the learning process.
I made a Dir en gray face classifier using TensorFlow --- ⑦ Learning model
I tried to visualize the model with the low-code machine learning library "PyCaret"
[Learning memo] Deep Learning made from scratch [Chapter 7]
Deep learning / Deep learning made from scratch Chapter 6 Memo
[Learning memo] Deep Learning made from scratch [Chapter 5]
[Learning memo] Deep Learning made from scratch [Chapter 6]
Deep learning / Deep learning made from scratch Chapter 7 Memo
[Learning memo] Deep Learning made from scratch [~ Chapter 4]
I tried to organize the evaluation indexes used in machine learning (regression model)
[Python] I tried the same calculation as LSTM predict with from scratch [Keras]