Machine learning memo of a fledgling engineer Part 2
Introduction
This is part 2 of the learning memo for "Deep-Learning from scratch".
2-layer neural network (Chapter 4)
- Read data from mnist.py. Normalize, one_hot array, and one-dimensional array of data.
train_neuralnet
#Data reading
(x_train, t_train), (x_test, t_test) = load_mnist(normalize=True,flatten=True, one_hot_label=True)
- Initialize the weight from two_layer_net.py. Generate keys for params {W1:, b1:, W2 :, b2:} in dictionary type. You can see the contents with print (network.params).
train_neuralnet
network = TwoLayerNet(input_size=784, hidden_size=50, output_size=10)
- Set each initial value and process 60,000 image data 100 images at a time.
train_neuralnet
iters_num = 10000 #Set the number of repetitions as appropriate
train_size = x_train.shape[0] # 60000
batch_size = 100
learning_rate = 0.1
train_loss_list = []
train_acc_list = []
test_acc_list = []
#Iterative processing per epoch 60000/ 100
iter_per_epoch = max(train_size / batch_size, 1)
- Randomly extract some data from the training data and perform learning. Here, 100 sheets are taken out from the data of 60,000 sheets.
train_neuralnet
for i in range(iters_num): #10000
#Get a mini batch
batch_mask = np.random.choice(train_size, batch_size) # (100,)Form of
x_batch = x_train[batch_mask] # (100,784)Form of
t_batch = t_train[batch_mask] # (100,784)Form of
#Gradient calculation
#grad = network.numerical_gradient(x_batch, t_batch)
grad = network.gradient(x_batch, t_batch)
#Parameter update
for key in ('W1', 'b1', 'W2', 'b2'):
network.params[key] -= learning_rate * grad[key]
loss = network.loss(x_batch, t_batch)
train_loss_list.append(loss)
#Save data when the conditions are met for 1 epoch at 600
if i % iter_per_epoch == 0:
train_acc = network.accuracy(x_train, t_train)
test_acc = network.accuracy(x_test, t_test)
train_acc_list.append(train_acc)
test_acc_list.append(test_acc)
print("train acc, test acc | " + str(train_acc) + ", " + str(test_acc))
- Draw a graph using the data saved in the list.
python
#Drawing a graph
x = np.arange(len(train_acc_list))
plt.plot(x, train_acc_list,'o', label='train acc')
plt.plot(x, test_acc_list, label='test acc', linestyle='--')
plt.xlabel("epochs")
plt.ylabel("accuracy")
plt.ylim(0, 1.0)
plt.legend(loc='lower right')
plt.show()
- Drawing result
reference
Deep Learning from scratch
Recommended Posts