Code for TensorFlow MNIST Begginer / Expert with Japanese comments

TensorFlow Tutorial MNIST code with Japanese comments written by many people was very useful during learning, so I also summarized my own. (If you find any mistakes, I'd be happy if you could point them out mm)

MNIST For ML Beginners

# coding: utf-8

#Data reading
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

#Load tensorflow as tf
import tensorflow as tf

# y = softmax(x*W + b)To define
##x[?, 784]Defined as a tensor of(None means I don't know how many)
x = tf.placeholder(tf.float32, [None, 784])

##W[784, 10]Defined as a tensor of. 784 pixels x each number from 0 to 9.
W = tf.Variable(tf.zeros([784, 10]))

##b[10](Initializewithall0s)Defined as a tensor of. Bias for each value from 0 to 9
b = tf.Variable(tf.zeros([10]))

##Apply each defined variable to the formula
##matmul is a function that integrates matrices
y = tf.nn.softmax(tf.matmul(x, W) + b)

# y_(label)To[?, 10]Defined as a tensor of
y_ = tf.placeholder(tf.float32, [None, 10])

# cross_Define entropy
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_  * tf.log(y), reduction_indices=[1]))

# cross_Define steps to optimize entropy using gradient descent
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

#Initialize variables
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)


#Perform learning in a mini-batch. Randomly take 100 pieces from the training data and repeat the operation to optimize 1000 times
for i in range(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict = {x: batch_xs, y_: batch_ys})

#Calculate the correct answer rate by comparing the prediction result with the label
##Get prediction results in matrix
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
##Calculate the hit rate
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

#Perform learning and output hit rate
print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))

capture

For basic tensorflow operations such as reduce_xxxxx processing, here is easy to understand.

Deep MNIST for Experts

This is easy to understand if you look at the network diagram first before looking at the code.

Screen Shot 2017-03-09 at 19.07.42.png

After defining all the networks, the calculation is performed by back-propagating at the end.


# coding: utf-8

#Load tensorflow
import tensorflow as tf

#Data reading
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data',one_hot=True)

#Launch an interactive session
sess = tf.InteractiveSession()

# x, y_Define placeholder
x = tf.placeholder(tf.float32,shape=[None,784])
y_ = tf.placeholder(tf.float32,shape=[None,10])

#Define function
##Weight initialization function
def weight_variable(shape):
    #Initialized with the left and right sides of the normal distribution cut out
    initial = tf.truncated_normal(shape,stddev=0.1)
    return tf.Variable(initial)


##Bias initialization function
def bias_variable(shape):
    initial = tf.constant(0.1,shape=shape)
    return tf.Variable(initial)


##Two-dimensional tatami mat prospect processing function(Stride 1,Zero padding)
def conv2d(x, W):
    return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding='SAME')


##Pooling function
def max_pool_2x2(x):
    return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')


#First time
##W used in the first convolution(Weighted value)
## ([width, height, input, filters]28 × 28 image 1(input)5 for each sheet(width)×5(height)32 types(filters)Fold each with the convolution filter of)
W_conv1 = weight_variable([5,5,1,32])
##Used in the first convolution b(bias)
b_conv1 = bias_variable([32])
#Input image. Returned to the tensor for convolution.
# -The value of 1 is dynamically determined at runtime by setting to reshape.
x_image = tf.reshape(x, [-1,28,28,1])
#1st convolution
h_conv1 = tf.nn.relu(conv2d(x_image,W_conv1) + b_conv1)
#1st pooling
h_pool1 = max_pool_2x2(h_conv1)


#Second time
##W used for the second convolution(Weighted value)
## ([width, height, input, filters]Convolve each of 32 14x14 images with 64 types of 5x5 convolution filters.)
W_conv2 = weight_variable([5,5,32,64])
##Used in the second convolution b(bias)
b_conv2 = bias_variable([64])
##Second convolution using the tensor output from the first pooling
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
##Second pooling
h_pool2 = max_pool_2x2(h_conv2)


#Fully connected layer(Put all the inputs together into the softmax function)
##W used in the fully connected layer(Weighted value)
W_fc1 = weight_variable([7*7*64,1024])
##Used in the fully connected layer b(bias)
b_fc1 = bias_variable([1024])
##H output in the second convolution_Convert pool2 to a two-dimensional tensor
h_pool2_flat = tf.reshape(h_pool2,[-1,7*7*64])
##Output the result in the fully connected layer
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1)


## DropOut(Processing to suppress overfitting)
keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1,keep_prob)

##W used for Drop Out to fully connected layer(Weighted value)
W_fc2 = weight_variable([1024,10])
##Used in Drop Out to fully connected layer b(bias)
b_fc2 = bias_variable([10])

#Output final forecast
y_conv = tf.matmul(h_fc1_drop, W_fc2) + b_fc2


#I want to minimize cross_Define entropy
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(y_conv,y_))
#Define minimization steps with Adam Optimizer
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

#This area is the same as Beginer
correct_prediction = tf.equal(tf.argmax(y_conv,1),tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
sess.run(tf.initialize_all_variables())

#Perform learning in a mini-batch(Randomly take 50 and repeat optimization 50 times)
for i in range(20000):
    batch = mnist.train.next_batch(50)
    #Output progress every 100 times
    if i%100 == 0:
        train_accuracy = accuracy.eval(feed_dict={x:batch[0],y_:batch[1],keep_prob:1.0})
        print("step %d, training accuracy %g"%(i,train_accuracy))
    train_step.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5})
    
print("test accuracy %g"%accuracy.eval(feed_dict={x: mnist.test.images, y_:mnist.test.labels, keep_prob: 1.0}))

capture

Here was very helpful for understanding convolutional neural networks.

Recommended Posts

Code for TensorFlow MNIST Begginer / Expert with Japanese comments
Try TensorFlow MNIST with RNN
I wrote the code for Japanese sentence generation with DeZero
[Roughly translate TensorFlow Tutorial into Japanese] 1. MNIST For ML Beginners
[Roughly translate TensorFlow Tutorial into Japanese] 2. Deep MNIST For Experts
MNIST (DCNN) with Keras (TensorFlow backend)
[Explanation for beginners] TensorFlow tutorial MNIST (for beginners)
TensorFlow MNIST For ML Beginners Translation
Classify "Wine" with TensorFlow MLP code
TensorFlow Tutorial MNIST For ML Beginners
I tried running the TensorFlow tutorial with comments (_TensorFlow_2_0_Introduction for beginners)
TensorFlow Tutorial -MNIST For ML Beginners
TensorFlow Deep MNIST for Experts Translation
[Explanation for beginners] TensorFlow tutorial Deep MNIST
Display Japanese graphs with VS Code + matplotlib
Supplementary notes for TensorFlow MNIST For ML Beginners
I rewrote Chainer's MNIST code with PyTorch + Ignite
Record of TensorFlow mnist expert edition (Visualization of TensorBoard)
Conducting the TensorFlow MNIST For ML Beginners Tutorial