Deep Learning / Deep Learning from Zero Chapter 3 Memo

1.First of all

I'm reading a masterpiece, ** "Deep Learning from Zero" **. This time is a memo of Chapter 3. To execute the code, download the entire code from Github and use jupyter notebook in ch03.

2.3 layer neural network

Consider a three-layer neural network. Assuming that the number of neurons is input layer 2, first layer 3, second layer 2, and output layer 2, forward propagation can be performed by the following matrix operation.

import numpy as np

def sigmoid(x):
    return 1/(1+np.exp(-x))

def identity_function(x):
    return x

def init_network():
    network={}
    network['W1']=np.array([[0.1, 0.3, 0.5],[0.2, 0.4, 0.6]])
    network['b1']=np.array([0.1, 0.2, 0.3])
    network['W2']=np.array([[0.1, 0.4],[0.2, 0.5],[0.3, 0.6]])
    network['b2']=np.array([0.1, 0.2])
    network['W3']=np.array([[0.1, 0.3],[0.2, 0.4]])
    network['b3']=np.array([0.1, 0.2])    
    return network

def forward(network, x):
    W1, W2, W3 = network['W1'], network['W2'], network['W3']
    b1, b2, b3 = network['b1'], network['b2'], network['b3']

    a1 = np.dot(x, W1)+b1
    z1 = sigmoid(a1)
    a2 = np.dot(z1, W2)+b2
    z2 = sigmoid(a2)
    a3 = np.dot(z2, W3)+b3
    y = identity_function(a3)    
    return y
        
network=init_network()
x = np.array([1.0, 0.5])
y = forward(network, x)
print(y)

#output
[0.31682708 0.69627909]

3. Inference with trained model

The text stores the trained parameters of the network that identify the numbers 0-9 (MNIST) and uses them to make inferences.

The number of neurons in the network is 784 in the input layer, 50 in the first layer, 100 in the second layer, and 10 in the output layer. As before, forward propagation can be expressed by the following matrix operation.

import sys, os
sys.path.append(os.pardir)  #Settings for importing files in the parent directory
import numpy as np
import pickle
from dataset.mnist import load_mnist
from common.functions import sigmoid, softmax

def get_data():
    (x_train, t_train), (x_test, t_test) = load_mnist(normalize=True, flatten=True, one_hot_label=False)
    return x_test, t_test

def init_network():
    with open("sample_weight.pkl", 'rb') as f:
        network = pickle.load(f)
    return network

def predict(network, x):
    W1, W2, W3 = network['W1'], network['W2'], network['W3']
    b1, b2, b3 = network['b1'], network['b2'], network['b3']

    a1 = np.dot(x, W1) + b1
    z1 = sigmoid(a1)
    a2 = np.dot(z1, W2) + b2
    z2 = sigmoid(a2)
    a3 = np.dot(z2, W3) + b3
    y = softmax(a3)
    return y

x, t = get_data()
network = init_network()
accuracy_cnt = 0
pred = []  #Prepare a list to save the inference result
for i in range(len(x)):
    y = predict(network, x[i])
    p = np.argmax(y) #Get the index of the most probable element
    pred.append(p)  #Save inference results
    if p == t[i]:
        accuracy_cnt += 1

print("Accuracy:" + str(float(accuracy_cnt) / len(x)))

#output
Accuracy:0.9352

Since I want to see the inference results individually later, I have added a list to save the inference results pred = [] and a save inference pred.append (p) to the code.

Now let's take a look at how we make individual predictions. Since the variable x contains the image and the variable pred contains the inference result,

import matplotlib.pyplot as plt

fig = plt.figure(figsize=(10, 12))
for i in range(50):
    ax = fig.add_subplot(10, 10, i+1, xticks=[], yticks=[])
    ax.imshow(x[i].reshape((28, 28)), cmap='gray')
    ax.set_xlabel('pred='+str(pred[i]))

The inference results of the 50 images from the beginning of the image data are displayed. I made a mistake only with the two images displayed in the red frame, which is quite excellent.

4. Batch processing

By the way, even if you do not do it one by one when inferring, it is efficient to batch process 100 sheets at once. In that case, you only need to change the inference part of the previous code.

batch_size = 100 #Number of batches
accuracy_cnt = 0
pred = np.array([])  #Box to store inference results(numpy)Prepare
for i in range(0, len(x), batch_size):
    x_batch = x[i:i+batch_size]
    y_batch = predict(network, x_batch)
    p = np.argmax(y_batch, axis=1)
    pred = np.append(pred, p) #Save inference results
    accuracy_cnt += np.sum(p == t[i:i+batch_size])

print("Accuracy:" + str(float(accuracy_cnt) / len(x)))
pred = pred.astype(int)  #Convert inference results from float to int

As before, we've added code to save the inference results. This time, the inference result is returned in numpy format for batch processing, so prepare the box to save pred = np.array ([]), save pred = np.append (pred, p) , If it is left as it is, it will be displayed as" 1.0 ", so to return it to an integer at the end,pred = pred.astype (int)is set.

Now, the point here is y_batch = predict (network, x_batch). This means that we put 100 data into the network input, so we get 100 outputs. Speaking of the image of matrix operation earlier,