Conducting the TensorFlow MNIST For ML Beginners Tutorial

Introduction

In Last time, I translated MNIST For ML Beginners, which is a tutorial of TensorFlow, so this time I will actually implement the contents of the tutorial using TensorFlow. I did. Even so, the code is already fluttering in the tutorial, but it's all out, so it's time to implement it while understanding what that code means.

Environmental setup

Since this is my first time using Python, I will also describe the setup so that even similar beginners can execute it. It's basically the same as what you see on the TensorFlow site. The target is mac.

Python installation

First, install Python itself.

brew install python

It also installs pip, Python's package management system.

sudo easy_install pip

It seems that TensorFlow recommends running in a Python virtual environment called virtualenv, so install it using pip.

sudo pip install --upgrade virtualenv

That's all for Python related installation.

Install TensorFlow

Before installing TensorFlow, configure the virtualenv environment settings. I'm not sure what --system-site-package means, but for the time being, follow the official website.

virtualenv --system-site-packages ./tensorflow

After completing the environment settings, run virtualenv.

cd tensorflow
source bin/activate

Install TensorFlow. Setup is now complete.

sudo pip install --upgrade https://storage.googleapis.com/tensorflow/mac/tensorflow-0.8.0-py2-none-any.whl

Run MNIST For ML Beginners

Now, here is the implementation and execution of the actual training.

Implementation code for MNIST For ML Beginners

The full code is below. Almost all the explanations are written in the comments in the code, so please take a look there.

I also gave the code to Gist. TensorFlow MNIST For ML Beginners Tutorial Code

mnist_for_ml_beginners.py


# -*- coding: utf-8 -*-

#Import TensorFlow
import tensorflow as tf
#Input to read MNIST_data.Put py in the same directory and import
# input_data.py has a link in the tutorial so get it from there
# https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/examples/tutorials/mnist/input_data.py
import input_data

import time

#Start time
start_time = time.time()
print "Start time: " + str(start_time)

#Read MNIST data
#60000 points of training data (mnist).train) and 10000 points of test data (mnist).test) is
#0 for training data and 0 for test data-9 images and their corresponding labels (0)-9) is
#The image is 28x28px(=784)Size
# mnist.train.images[60000, 784]Is an array of mnist.train.lables[60000, 10]Array of
#The lables array is if the corresponding images image is a number of 3.[0,0,0,1,0,0,0,0,0,0]Has become
# mnist.test.images[10000, 784]Is an array of mnist.test.lables[10000, 10]Array of
print "---Start reading MNIST data---"
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
print "---Completion of reading MNIST data---"

#Variable to put the training image
#The training image is 28x28px, and these are sorted and stored in a vector of 1 row and 784 columns.
#None is set so that any number of training images can be inserted.
x = tf.placeholder(tf.float32, [None, 784])

#weight
#Lines of px number of training image, label (0-Number of numbers 9) Matrix of columns of numbers
#Enter 0 as the initial value
W = tf.Variable(tf.zeros([784, 10]))

#bias
#Matrix of label number columns
#Enter 0 as the initial value
b = tf.Variable(tf.zeros([10]))

#Perform softmax regression
#y is the distribution of the probabilities that it is a number with respect to the input x (image)
#After multiplying the matrices x and W with the matmul function, add b.
#y is[1, 10]Matrix
y = tf.nn.softmax(tf.matmul(x, W) + b)

#Cross entropy
# y_Is the label of the correct answer data
y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = -tf.reduce_sum(y_*tf.log(y))

#Optimize y to minimize cross entropy using gradient hardening
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)

#Execute initialization of the prepared variable Veriable
init = tf.initialize_all_variables()

#Start Session
#Execution starts for the first time by running (run(init)Otherwise init will not be executed)

sess = tf.Session()
sess.run(init)

#1000 trainings (train)_step)
# next_batch(100)Select 100 random training sets (images and corresponding labels) with
#Since there are 60,000 training data, I would like to use all of them, but since it costs money, that is, it takes time, I use 100 random ones
#You can get similar results with 100
# feed_You can enter a value in placeholder with dict
print "---Start training---"
for i in range(1000):
	batch_xs, batch_ys = mnist.train.next_batch(100)
	sess.run(train_step, feed_dict={x: batch_xs, y_:batch_ys})
print "---Training end---"

#Prediction of correctness
#Prediction y and correct label y of which number the calculated image is_To compare
#Returns True if they are the same
#argmax returns the index of the largest value in the array
#The index with the highest value means that it has the highest probability of being that number.
#If True is returned, it means that the result of training is the same as the answer.
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))

#Accuracy calculation
# correct_Since prediction is boolean, cast it to float and calculate the mean value
#Converted to 1 if True, 0 if False
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

#Execution and display of precision
#Check the accuracy with the image and label of the test data
#Since the values of W and b are calculated by softmax regression, y can be calculated by inputting x.
print "accuracy"
print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))

#End time
end_time = time.time()
print "End time: " + str(end_time)
print "Time taken: " + str(end_time - start_time)

The second argument of tf.argmax seems to specify the number of dimensions, but since the array of y and y_ fetches one from it in 1 row and 10 columns, specify 1 (dimension). Is it there?

Execution result

The execution itself was very quick and I was able to complete from training to confirmation in less than 3 seconds. When I first ran it, it took me about 5 minutes to download the MNIST data.

The execution method and output result are as follows.

$ python mnist_for_ml_beginners.py
Start time: 1449994007.63
---Start reading MNIST data---
Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
---Completion of reading MNIST data---
can't determine number of CPU cores: assuming 4
I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 4
can't determine number of CPU cores: assuming 4
I tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 4
---Start training---
---Training end---
accuracy
0.9212
End time: 1449994010.09
Time taken: 2.45791196823

You can see that the accuracy is over 92%. Since the result of the input image does not show what was output, it is just a numerical result. This accuracy will vary from run to run. This is because the training data used during training is different.

After finishing, deactivate virtualenv to finish.

deactivate

in conclusion

When I translated the tutorial last time, I checked the contents to some extent, so I was able to proceed while understanding that this code is like this when implementing this time. I still don't understand softmax regression and cross entropy, so I think it's better to study this area as well. As the name suggests, Softmax regression is also used for regression analysis.

This time I ran the tutorial as it is, so I just used the one with the image prepared, but I would like to try another image etc. However, in that case, I think that it is necessary to adjust the image size, normalize it, see the contents of input_data.py and understand it, so it will take a while, so next is [Tutorial for experts](https:: //www.tensorflow.org/versions/master/tutorials/mnist/pros/index.html#deep-mnist-for-experts) I would like to try it. Is this the production? It's like Deep Learning.

Reference site for setup

TensorFlow Download and Setup I tried running Hello World with TensorFlow & its explanation

Recommended Posts

Conducting the TensorFlow MNIST For ML Beginners Tutorial
TensorFlow Tutorial MNIST For ML Beginners
TensorFlow Tutorial -MNIST For ML Beginners
[Explanation for beginners] TensorFlow tutorial MNIST (for beginners)
TensorFlow MNIST For ML Beginners Translation
I tried the MNIST tutorial for beginners of tensorflow.
[Explanation for beginners] TensorFlow tutorial Deep MNIST
[Roughly translate TensorFlow Tutorial into Japanese] 1. MNIST For ML Beginners
Supplementary notes for TensorFlow MNIST For ML Beginners
I tried running the TensorFlow tutorial with comments (_TensorFlow_2_0_Introduction for beginners)
[Translation] NumPy Official Tutorial "NumPy: the absolute basics for beginners"
[Roughly translate TensorFlow Tutorial into Japanese] 2. Deep MNIST For Experts
[For beginners] I tried using the Tensorflow Object Detection API
I tried the TensorFlow tutorial 1st
I tried the TensorFlow tutorial 2nd
[Deprecated] Chainer v1.24.0 Tutorial for beginners
TensorFlow Deep MNIST for Experts Translation
I tried tensorflow for the first time
Challenges for the Titanic Competition for Kaggle Beginners
Django tutorial summary for beginners by beginners ③ (View)
Beginners read "Introduction to TensorFlow 2.0 for Experts"
TensorFlow tutorial tutorial
Django tutorial summary for beginners by beginners ⑤ (test)
Visualization of the firing state of the hidden layer of the model learned in the TensorFlow MNIST tutorial
[Explanation for beginners] TensorFlow basic syntax and concept
The fastest way for beginners to master Python
Django tutorial summary for beginners by beginners ⑦ (Customize Admin)
Django tutorial summary for beginners by beginners ⑥ (static file)
Django Tutorial Summary for Beginners by Beginners (Model, Admin)
Django tutorial summary for beginners by beginners ① (project creation ~)
Installing TensorFlow on Windows Easy for Python beginners
Roadmap for beginners
Mathematics for ML
Code for TensorFlow MNIST Begginer / Expert with Japanese comments
[For beginners] Install the package in the Anaconda environment (Janome)
MNIST image generation program creation by DCGAN (tensorflow tutorial)
[For beginners] Quantify the similarity of sentences with TF-IDF