Introduction to Private Chainer I've tried it, and I'll continue. However, rather than going in the direction of mastering chainer, I want to mess things up. Well, that's why it's TensorFlow.

Please refer to MNIST For ML Beginners.

MNIST in Introduction to Private Chainer and Handwriting recognition using KNN in Python However, using this to "make a number discriminator" is the basis of the machine learning tutorial. In other words, it's like a person who is new to the Oracle database enters from the SQL statement that selects the scott / tiger emp database.

From the preparation of TensorFlow

Use TensorFlow from Python. Since the Python environment is created with Anaconda, install it as follows according to Documentation.

% conda install -c conda-forge tensorflow

This completes the installation.

Return to MNIST For ML Beginners

Execute the following code without knowing the reason.

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

For the time being, when I ran it with ipython, it became as follows.

In [1]: from tensorflow.examples.tutorials.mnist import input_data

In [2]: mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting MNIST_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz

Apparently, the data has been downloaded.

% % ls -s1 MNIST_data/
total 22672
 3224 t10k-images-idx3-ubyte.gz
   16 t10k-labels-idx1-ubyte.gz
19368 train-images-idx3-ubyte.gz
   64 train-labels-idx1-ubyte.gz

MNIST implementation in TensorFlow

According to MNIST For ML Beginners, the 28x28 pixel handwritten digit image is either 0-9, as was the chainer tutorial. Is there? Is evaluated by the linear equation of y = Wx + b. The difference is that in the chainer tutorial, what was chainer.links.Linear () should be evaluated using Softmax Regression. I don't understand the difference in this area because I haven't studied enough.

According to Tutorial, the image data is 28x28 pixels per number, but this is one-dimensional 28x28 = 784 array data. It seems that it is assumed that there are 55,000 of them. That is, it is assumed that all the data is a matrix of 784 rows × 55000 columns, and this is mnist.train.xs. The label of each column (that is, which indicates which column is 0 to 9?) Is held as a matrix of 10 rows × 55000 columns, and the sole is mnist.train.ys. Why 10 lines? Speaking of which, 0 is [1,0,0,0,0,0,0,0,0,0] and 3 is [0,0,0,1,0,0,0,0,0,0]. It seems that it is because it is expressed as.

Data reading and formula definition

import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 784])

First, define the data of 784 columns as a placeholder.

W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))

Define the parameters to be learned as Variable.

Then define the model.

y = tf.nn.softmax(tf.matmul(x, W) + b)

matmul () is a matrix multiplication. Here, y can be obtained by softmax regression. Note that y is a 10-row matrix shown as follows.

<tf.Tensor 'Softmax_1:0' shape=(?, 10) dtype=float32>

Preparing for optimization

y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))

By the way, I think that optimization is the least squares method, but apparently it uses the cross_entropy function called -Σy'log (y). So, in order to minimize this cross_entropy, [gradient descent algorithm](https://ja.wikipedia.org/wiki/%E6%9C%80%E6%80%A5%E9%99%8D%E4 % B8% 8B% E6% B3% 95) is used. This is very similar to the SGD we called in the chainer tutorial, but it's a different approach.

train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

Now, initialize the variables and get ready for execution. The definition of variable initialization, but according to the Tutorial

`init = tf.global_variables_initializer()`


 It seems that it should be written, but in reality this is an error.

Traceback (most recent call last): File "./mnist_train_tf.py", line 14, in init = tf.global_variables_initializer() AttributeError: module 'tensorflow' has no attribute 'global_variables_initializer'


 Apparently, according to https://github.com/tensorflow/tensorflow/issues/5514#issuecomment-260840208, it seems that `` `global_variables_initializer ()` `` was defined from r0.12 of tensorflow, before r0.11. Seems to have used ```initialize_all_variables ()` ``. The tutorial is r0.12 compliant, but the tensorflow installed in the anaconda environment is r0.11, so it seems that an error occurred.

## Trial for optimization
 Define a Session with tensorflow, and select 100 from the test data in that session to learn.
 That is repeated 1000 times.

```python
sess = tf.Session()
sess.run(init)
for i in range(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

The train_step defined above is for minimizing cross_entropy, and the data batch_xs and label batch_ys obtained from mnist are input to it via the argument feed_dict.

Execution result

Since it is 0.9184, I think it is unexpectedly low as a numerical value.

Tutorial also says like this.

This should be about 92%.

Is that good? Well, not really. In fact, it's pretty bad. This is because we're using a very simple model. With some small changes, we can get to 97%. The best models can get to over 99.7% accuracy!

Well, it seems that it's because I used a simple model for the tutorial.

Today's summary

--I tried using TensorFlow --It was different from chainer --There was a method with different names in release 0.12 and 0.11

It turned out that even with the neural network approach, there are completely different implementation methods for each tool. I also found that I didn't have enough knowledge about neural networks and artificial intelligence to understand this code properly.

Today's code

https://github.com/fukuit/Python_SelfLearning/blob/master/tensorflow_lesson/mnist_train_tf.py

2017/02/22 postscript

With the release of TensorFlow 1.0, the installation method has changed. Try it on your Windows machine.

Installation on Windows

It says to install with pip instead of conda at https://www.tensorflow.org/install/install_windows.

that?

For the time being, when I run the sample code to check if it was installed, it works, but I get a lot of warnings.

>>> import tensorflow as tf
>>> hello = tf.constant('Hello, Tensorflow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "BestSplits" device_type: "CPU"') for unknown op: BestSplits
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "CountExtremelyRandomStats" device_type: "CPU"') for unknown op: Cou
ntExtremelyRandomStats
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "FinishedNodes" device_type: "CPU"') for unknown op: FinishedNodes
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "GrowTree" device_type: "CPU"') for unknown op: GrowTree
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "ReinterpretStringToFloat" device_type: "CPU"') for unknown op: Rein
terpretStringToFloat
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "SampleInputs" device_type: "CPU"') for unknown op: SampleInputs
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "ScatterAddNdim" device_type: "CPU"') for unknown op: ScatterAddNdim

E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "TopNInsert" device_type: "CPU"') for unknown op: TopNInsert
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "TopNRemove" device_type: "CPU"') for unknown op: TopNRemove
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "TreePredictions" device_type: "CPU"') for unknown op: TreePredictio
ns
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "UpdateFertileSlots" device_type: "CPU"') for unknown op: UpdateFert
ileSlots
b'Hello, Tensorflow!'

Hello Tensorflow! Is displayed at the end, but what is this on the way?

An introduction to private TensorFlow