Machine learning memo of a fledgling engineer Part 1

Introduction

This is part 1 of the learning memo of "Deep-Learning made from scratch".

Slice operation that is easy to forget

• You can operate the numerical value displayed by [start: step: stop].

slice

x=[1,2,3,4,5]
#Full display
x[:]
#1 to 2
x[0:2]
#Skip two of 1, 3 and 5
x[::2]
#Reverse order,-If set to 2, skip two from the opposite
x[::-1]

Numpy shape match

• If this is not known, data preprocessing cannot be performed.

numpy

#3x2 shape
A = np.array([[1,2], 
              [3,4],
              [5,6]])
#2 x 1 shape
B = np.array([7,
              8])
#Inner product calculation
np.dot(A,B)
>>>array([23,53,83])

Neural network implementation (Chapter 3)

• Since you are using Google Colaboratory, you need to mount Google Drive, otherwise you will not be able to import another Python file.

mount

from google.colab import drive
drive.mount('/content/drive')

cd drive/My Drive/deep-learning-from-scratch-master/ch03

• Calling import statement, numpy, pickle, function in another file

neuralnet

# coding: utf-8
import sys, os
sys.path.append(os.pardir)  #Settings for importing files in the parent directory
import numpy as np
import pickle
from dataset.mnist import load_mnist
from common.functions import sigmoid, softmax

• Training data and training label and test data and test label are called and stored from mnist.py, and only x_test and t_test are called here because inference processing is performed using the learned data. Training data is not used because it is not for learning.

neuralnet

def get_data():
    (x_train, t_train), (x_test, t_test) = load_mnist(normalize=True, flatten=True, one_hot_label=False)
    return x_test, t_test

• "Sample_weight.pkl" is the trained weight data, and the content of the data is dictionary type and contains {W1:, W2 :, W3 :, b1:, b2 :, b3}.

neuralnet

def init_network():
    with open("sample_weight.pkl", 'rb') as f:
        network = pickle.load(f)
    return network

• Store the data called from "sample_weight.pkl" in a variable, put it in the model, and the result will come out as a Numpy array. At this time, the largest number in the numpy array is the predicted number.

neuralnet

def predict(network, x):
    W1, W2, W3 = network['W1'], network['W2'], network['W3']
    b1, b2, b3 = network['b1'], network['b2'], network['b3']

    a1 = np.dot(x, W1) + b1
    z1 = sigmoid(a1)
    a2 = np.dot(z1, W2) + b2
    z2 = sigmoid(a2)
    a3 = np.dot(z2, W3) + b3
    y = softmax(a3)

    return y

• The variable t contains label data, and the if statement determines whether the predicted value and the correct label match. Here, it is judged how many correct answers out of 10,000 image data. The result is Accuracy: 0.9352.

neuralnet

x, t = get_data()
network = init_network()
accuracy_cnt = 0
for i in range(len(x)):
    y = predict(network, x[i])
    p= np.argmax(y) #Get the index of the most probable element
    if p == t[i]:
        accuracy_cnt += 1

print("Accuracy:" + str(float(accuracy_cnt) / len(x)))

Batch processing (continued in Chapter 3)

• The neural network described in the previous item consists of neurons with 784 input layers and 10 output layers. 784 is the image size of 28 × 28, and 10 is the classification of the numbers 0 to 9. The setting is such that there are two hidden layers, the first hidden layer has 50 neurons, and the second hidden layer has 100 neurons. This matches the form of the trained data "sample_weight.pkl".
• In batch processing, 100 images out of 10000 image data can be collectively used as input data. This speeds up the process.
• The part where axis = 1 is set to take the maximum value in the column direction.
• In np.sum (p == t [i: i + batch_size]), True is judged by the comparison operator (==) and the number is calculated.

Batch processing

x, t = get_data()
network = init_network()

batch_size = 100 #Number of batches
accuracy_cnt = 0

#range(start,stop,step)
for i in range(0, len(x), batch_size):
    x_batch = x[i:i+batch_size]
    y_batch = predict(network, x_batch)
    p = np.argmax(y_batch, axis=1)
    accuracy_cnt += np.sum(p == t[i:i+batch_size])

print("Accuracy:" + str(float(accuracy_cnt) / len(x)))

reference

Deep Learning from scratch