I tried a TensorFlow tutorial (MNIST for beginners) on Cloud9-Classification of handwritten images-

Introduction

I tried "MNIST For ML Beginners" which is written for beginners of machine learning in the TensorFlow tutorial. So I implemented MNIST's number recognition, which is written as Hello World of machine learning. MNIST is a data set of handwritten images. Here, images of handwritten numbers from 0 to 9 are read and classified by opportunity learning. The sample code on Github of TensorFlow seems to be difficult, but when I tried only the necessary implementation, it was really simple and I could implement it in 20 lines.

environment

Cloud9 Python 2.7.6 Sample Codes : GitHub Environment construction is "Use TensorFlow in cloud integrated development environment Cloud9 ~ GetStarted ~" The basic usage of TesorFlow is "Use TensorFlow in cloud integrated development environment Cloud9-Basics of usage-" See

Learning process code

It is located on Github, but the following is the implemented code.

mnist_softmax_train.py


from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf

# Download gz files to MNIST_data directory
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)

# Initializing
sess = tf.InteractiveSession()

x = tf.placeholder(tf.float32, shape=[None, 28*28])
y_ = tf.placeholder(tf.float32, shape=[None, 10])

W = tf.Variable(tf.zeros([28*28, 10]), name="W")
b = tf.Variable(tf.zeros([10]), name="b")

sess.run(tf.initialize_all_variables())

# Making model
y = tf.matmul(x, W) + b
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y, y_))

# Training
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
for i in range(1000):
  batch = mnist.train.next_batch(100)
  train_step.run(feed_dict={x: batch[0], y_: batch[1]})

# Evaluating
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels}))

# Save train data
saver = tf.train.Saver()
saver.save(sess, 'param/softmax.param')

When I run it, it seems that the accuracy is about 92% as described in the tutorial.

The contents of the code

mnist = input_data.read_data_sets('MNIST_data', one_hot=True)

I'm getting the data. I created a directory called MNIST_data and downloaded 4 files there. The contents are a black box, so you need to check it.

sess = tf.InteractiveSession()

x = tf.placeholder(tf.float32, shape=[None, 28*28])
y_ = tf.placeholder(tf.float32, shape=[None, 10])

W = tf.Variable(tf.zeros([28*28, 10]), name="W")
b = tf.Variable(tf.zeros([10]), name="b")

sess.run(tf.initialize_all_variables())

This is the initialization process. x holds the image data (28 pixels x 28 pixels) and y_ holds the label (0-9) of the image. Since it is a placeholder, I will assign it later. W and b are parameters, where W is the weight of each pixel in the image and b is the intercept.

y = tf.matmul(x, W) + b
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y, y_))

y is an expression that calculates the predicted value from the parameter and x. The following equation is the softmax regression. Logistic regression is a model that predicts 0 and 1, but it is extended to be able to predict multiple labels (0-9 in this case). The tutorial also included detailed formulas, so I think you need to study that as well.

train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
for i in range(1000):
  batch = mnist.train.next_batch(100)
  train_step.run(feed_dict={x: batch[0], y_: batch[1]})

Gradient Descent Optimizer is a stochastic gradient descent method. The first downloaded data with mnist.train.next_batch is acquired in order and assigned to the variable with feed_dict = {x: batch [0], y_: batch [1]}. We are looking for the optimal solution by repeatedly executing it.

correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels}))

We are using test data to check the accuracy of y (predicted value) and y_ (actual value).

saver = tf.train.Saver()
saver.save(sess, 'param/softmax.param')

This is the process of saving the learned parameters. You can now classify the images later using only the parameters.

Predicting your handwritten data

You learned the handwritten numerical data above. Using that parameter, I tried to classify the handwritten data I wrote. My handwritten data is also on Github, but it's a black and white bmp. I'm loading it with parsebmp.py. The MNIST data is learned with a numerical value from 0 to 1, but this process is easy with only 0 and 1 data. When I actually moved it, it was 60% accurate. It's far from 92%, but it can be said that it can be judged to some extent. ⇒ Preprocessing was required to predict handwritten data. For more information, please refer to the article Predicting your handwritten data with TensorFlow.

mnist_softmax.py


from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf
import sys
import numpy as np
import parsebmp as pb

def whatisit(file, sess):
  print("File name is %s" % file)
  data = pb.parse_bmp(file)

  # Show bmp data
  for i in range(len(data)):
    sys.stdout.write(str(int(data[i])))
    if (i+1) % 28 == 0:
      print("")

  # Predicting
  d = np.array([data])
  result = sess.run(y, feed_dict={x: d})

  # Show result
  print(result)
  print(np.argmax(result, 1))


if __name__ == "__main__":
  # Restore parameters
  x = tf.placeholder(tf.float32, shape=[None, 28*28])
  y_ = tf.placeholder(tf.float32, shape=[None, 10])
  W = tf.Variable(tf.zeros([28*28, 10]), name="W")
  b = tf.Variable(tf.zeros([10]), name="b")
  y = tf.matmul(x, W) + b
  
  sess = tf.InteractiveSession()
  saver = tf.train.Saver()
  saver.restore(sess, 'param/softmax.param')

  # My data
  whatisit("My_data/0.bmp", sess)
  whatisit("My_data/1.bmp", sess)
  whatisit("My_data/2.bmp", sess)
  whatisit("My_data/3.bmp", sess)
  whatisit("My_data/4.bmp", sess)
  whatisit("My_data/5.bmp", sess)
  whatisit("My_data/6.bmp", sess)
  whatisit("My_data/7.bmp", sess)
  whatisit("My_data/8.bmp", sess)
  whatisit("My_data/9.bmp", sess)

in conclusion

First, I implemented the code while watching the tutorial. It's really easy to implement, but I think you need to understand the following points.

--Contents of usage data --Detailed formula for softmax regression

Also, in order to really understand it, I would like to be able to take in my own handwritten data and make predictions. I think that will lead to practical use. ⇒ I tried to predict my handwritten data on 2017/03/27.

Change log

--2018/06/12: Added about prediction of handwritten data --2017/03/27: Added how to save parameters and predicted contents of your own handwritten data --2016/10/30: New post

Recommended Posts

I tried a TensorFlow tutorial (MNIST for beginners) on Cloud9-Classification of handwritten images-
I tried the MNIST tutorial for beginners of tensorflow.
I tried a convolutional neural network (CNN) with a tutorial on TensorFlow on Cloud9-Classification of handwritten images-
[Explanation for beginners] TensorFlow tutorial MNIST (for beginners)
TensorFlow Tutorial MNIST For ML Beginners
TensorFlow Tutorial -MNIST For ML Beginners
[Explanation for beginners] TensorFlow tutorial Deep MNIST
I tried running the TensorFlow tutorial with comments (_TensorFlow_2_0_Introduction for beginners)
Conducting the TensorFlow MNIST For ML Beginners Tutorial
I made a demo that lets the model learned in the Tensorflow mnist tutorial distinguish the handwritten numbers written on the canvas.
[Roughly translate TensorFlow Tutorial into Japanese] 1. MNIST For ML Beginners
[For beginners] I tried using the Tensorflow Object Detection API
TensorFlow MNIST For ML Beginners Translation
I tried the TensorFlow tutorial 1st
I tried the TensorFlow tutorial 2nd
I tried TensorFlow tutorial CNN 4th
I tried object detection with YOLO v3 (TensorFlow 2.0) on a windows CPU!
I tried tensorflow for the first time
I tried playing a ○ ✕ game using TensorFlow
Supplementary notes for TensorFlow MNIST For ML Beginners
I built a TensorFlow environment on windows10
I tried 3D detection of a car
I tried running the TensorFlow tutorial with comments (text classification of movie reviews)
I tried object detection with YOLO v3 (TensorFlow 2.1) on the GPU of windows!
A memo of a tutorial on running python on heroku
Installing TensorFlow on Windows Easy for Python beginners
I tried to make a ○ ✕ game using TensorFlow
I tried to make something like a chatbot with the Seq2Seq model of TensorFlow
I want to collect a lot of images, so I tried using "google image download"
Create a dataset of images to use for learning
I tried Python on Mac for the first time.
I tried python on heroku for the first time
Preparation for processing pbm images on a pixel-by-pixel basis
What I did when I stumbled on a Django tutorial
A memo of installing Chainer 1.5 for GPU on Windows
I tried a visual regression test on GitHub Pages
A note of trying a simple MCMC tutorial on PyMC3
I tried a simple RPA for login with selenium
I tried using Python (3) instead of a scientific calculator
I tried Kaokore, a Japanese classic dataset, on EfficientNet.
I tried "morphology conversion" of images with Python + OpenCV
I tried running TensorFlow
I thought about a Python beginner's course on blockchain games
I tried to find the average of the sequence with TensorFlow
I tried refactoring the CNN model of TensorFlow using TF-Slim
I tried running PIFuHD on Windows for the time being
I wrote a demo program for linear transformation of a matrix
I tried using Tensorboard, a visualization tool for machine learning
I made a VGG16 model using TensorFlow (on the way)
I tried a little bit of the behavior of the zip function
[Roughly translate TensorFlow Tutorial into Japanese] 2. Deep MNIST For Experts
I tried to create a bot for PES event notification
[For beginners] A word summary of popular programming languages (2018 version)
I stumbled on TensorFlow (What is Out of GPU Memory)
I tried a stochastic simulation of a bingo game with Python