"Deep Learning from scratch" Self-study memo (No. 17) I tried to build DeepConvNet with Keras

While reading "Deep Learning from scratch" (written by Yasuki Saito, published by O'Reilly Japan), I will make a note of the sites I referred to. Part 16

DeepConvNet Let's build DeepConvNet with Keras as described on page 241 of the book.

g8-1.jpg

I'm wondering if there are a lot of layers and it's deep there, but I'm not sure why this improves recognition accuracy.

But

You can imitate the sample script and run it.

I tried it.

from google.colab import drive
drive.mount('/content/drive')

import sys, os
sys.path.append('/content/drive/My Drive/Colab Notebooks/deep_learning/common')
sys.path.append('/content/drive/My Drive/Colab Notebooks/deep_learning/dataset')

#TensorFlow and tf.import keras
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from keras.layers import Dense, Activation, Flatten, Conv2D, MaxPooling2D, Dropout

#Import helper library
import numpy as np
import matplotlib.pyplot as plt

from mnist import load_mnist
#Data reading
(x_train, t_train), (x_test, t_test) = load_mnist(flatten=False)
X_train = x_train.transpose(0,2,3,1)
X_test = x_test.transpose(0,2,3,1)
input_shape=(28,28,1)
filter_num = 16
filter_size = 3
filter_stride = 1
filter_num2 = 32
filter_num3 = 64
pool_size_h=2
pool_size_w=2
pool_stride=2
d_rate = 0.5
hidden_size=100
output_size=10

model = keras.Sequential(name="DeepConvNet")
model.add(keras.Input(shape=input_shape))
model.add(Conv2D(filter_num, filter_size, strides=filter_stride, padding="same", activation="relu", kernel_initializer='he_normal'))
model.add(Conv2D(filter_num, filter_size, strides=filter_stride, padding="same", activation="relu", kernel_initializer='he_normal'))
model.add(MaxPooling2D(pool_size=(pool_size_h, pool_size_w),strides=pool_stride))

model.add(Conv2D(filter_num2, filter_size, strides=filter_stride, padding="same", activation="relu", kernel_initializer='he_normal'))
model.add(Conv2D(filter_num2, filter_size, strides=filter_stride, padding="same", activation="relu", kernel_initializer='he_normal'))
model.add(MaxPooling2D(pool_size=(pool_size_h, pool_size_w),strides=pool_stride))

model.add(Conv2D(filter_num3, filter_size, strides=filter_stride, padding="same", activation="relu", kernel_initializer='he_normal'))
model.add(Conv2D(filter_num3, filter_size, strides=filter_stride, padding="same", activation="relu", kernel_initializer='he_normal'))
model.add(MaxPooling2D(pool_size=(pool_size_h, pool_size_w),strides=pool_stride))

model.add(keras.layers.Flatten())
model.add(Dense(hidden_size, activation="relu", kernel_initializer='he_normal')) 
model.add(Dropout(d_rate))
model.add(Dense(output_size))
model.add(Dropout(d_rate))
model.add(Activation("softmax")) 

#Compiling the model
model.compile(loss="sparse_categorical_crossentropy", 
              optimizer="adam", 
              metrics=["accuracy"])

By specifying padding = "same", the output image will be the same size as the input image.

model.summary()

Model: "DeepConvNet" Layer (type)          Output Shape      Param #


conv2d (Conv2D)        (None, 28, 28, 16)    160
conv2d_1 (Conv2D)       (None, 28, 28, 16)    2320
max_pooling2d (MaxPooling2D) (None, 14, 14, 16)    0
conv2d_2 (Conv2D)       (None, 14, 14, 32)    4640
conv2d_3 (Conv2D)       (None, 14, 14, 32)    9248
max_pooling2d_1 (MaxPooling2 (None, 7, 7, 32)     0
conv2d_4 (Conv2D)       (None, 7, 7, 64)     18496
conv2d_5 (Conv2D)       (None, 7, 7, 64)     36928
max_pooling2d_2 (MaxPooling2 (None, 3, 3, 64)     0
flatten (Flatten)       (None, 576)        0
dense (Dense)         (None, 100)        57700
dropout (Dropout)       (None, 100)        0
dense_1 (Dense)        (None, 10)         1010
dropout_1 (Dropout)      (None, 10)        0
activation (Activation)    (None, 10)        0


Total params: 130,502 Trainable params: 130,502 Non-trainable params: 0

model.fit(X_train, t_train,  epochs=5, batch_size=128)
test_loss, test_acc = model.evaluate(X_test,  t_test, verbose=2)
print('\nTest accuracy:', test_acc)

313/313 - 6s - loss: 0.0313 - accuracy: 0.9902 Test accuracy: 0.9901999831199646

It seems to be working properly.

Part 16 ← Click here for the table of contents of the memo Unreadable Glossary

Recommended Posts

"Deep Learning from scratch" Self-study memo (No. 17) I tried to build DeepConvNet with Keras
"Deep Learning from scratch" Self-study memo (No. 16) I tried to build SimpleConvNet with Keras
"Deep Learning from scratch" Self-study memo (No. 11) CNN
"Deep Learning from scratch" Self-study memo (No. 19) Data Augmentation
"Deep Learning from scratch 2" Self-study memo (No. 21) Chapters 3 and 4
[Deep Learning from scratch] I tried to explain Dropout
"Deep Learning from scratch" self-study memo (No. 18) One! Meow! Grad-CAM!
"Deep Learning from scratch" self-study memo (No. 19-2) Data Augmentation continued
I tried to implement Perceptron Part 1 [Deep Learning from scratch]
"Deep Learning from scratch" self-study memo (No. 15) TensorFlow beginner tutorial
"Deep Learning from scratch" Self-study memo (Part 12) Deep learning
I tried to make deep learning scalable with Spark × Keras × Docker
"Deep Learning from scratch" self-study memo (No. 13) Try using Google Colaboratory
"Deep Learning from scratch" Self-study memo (No. 10-2) Initial value of weight
"Deep Learning from scratch" Self-study memo (Part 8) I drew the graph in Chapter 6 with matplotlib
"Deep Learning from scratch" Self-study memo (9) MultiLayerNet class
"Deep Learning from scratch" Self-study memo (10) MultiLayerNet class
[Deep Learning from scratch] I tried to implement sigmoid layer and Relu layer.
I tried to extract a line art from an image with Deep Learning
I tried to make deep learning scalable with Spark × Keras × Docker 2 Multi-host edition
Try to build a deep learning / neural network with scratch
I tried to divide with a deep learning language model
[Deep Learning from scratch] I tried to explain the gradient confirmation in an easy-to-understand manner.
"Deep Learning from scratch" Self-study memo (No. 14) Run the program in Chapter 4 on Google Colaboratory
[Learning memo] Deep Learning made from scratch [Chapter 7]
[Learning memo] Deep Learning made from scratch [Chapter 5]
[Learning memo] Deep Learning made from scratch [Chapter 6]
Deep learning / Deep learning made from scratch Chapter 7 Memo
[Learning memo] Deep Learning made from scratch [~ Chapter 4]
I tried to implement deep learning that is not deep with only NumPy
I tried to move GAN (mnist) with keras
Deep Learning from scratch ① Chapter 6 "Techniques related to learning"
Deep Learning from scratch Chapter 2 Perceptron (reading memo)
I tried to integrate with Keras in TFv1.1
[Learning memo] Deep Learning from scratch ~ Implementation of Dropout ~
Deep Learning from scratch
I'm not sure, but I feel like I understand Deep Learning (I tried Deep Learning from scratch)
[Python] [Natural language processing] I tried Deep Learning ❷ made from scratch in Japanese ①
I tried deep learning
I tried to implement Cifar10 with SONY Deep Learning library NNabla [Nippon Hurray]
[Python] I tried the same calculation as LSTM predict with from scratch [Keras]
[Deep Learning from scratch] I implemented the Affine layer
I tried to build ML Pipeline with Cloud Composer
I tried to implement Grad-CAM with keras and tensorflow
I tried to predict horse racing by doing everything from data collection to deep learning
Dare to learn with Ruby "Deep Learning from scratch" Importing pickle files from forbidden PyCall
I tried to build an environment for machine learning with Python (Mac OS X)
Create an environment for "Deep Learning from scratch" with Docker
Mayungo's Python Learning Episode 3: I tried to print numbers with print
I tried to implement ListNet of rank learning with Chainer
I captured the Touhou Project with Deep Learning ... I wanted to.
I tried to make Othello AI that I learned 7.2 million hands by deep learning with Chainer
A memo when executing the deep learning sample code created from scratch with Google Colaboratory
Deep Learning / Deep Learning from Zero 2 Chapter 4 Memo
I tried to implement Deep VQE
Deep Learning / Deep Learning from Zero 2 Chapter 5 Memo
[Python] A memo that I tried to get started with asyncio
Deep learning from scratch (cost calculation)
I tried deep reinforcement learning (Double DQN) for tic-tac-toe with ChainerRL
I read "Reinforcement Learning with Python: From Introduction to Practice" Chapter 1
Deep Learning / Deep Learning from Zero 2 Chapter 7 Memo