An introduction to private TensorFlow

Introduction to Private Chainer I've tried it, and I'll continue. However, rather than going in the direction of mastering chainer, I want to mess things up. Well, that's why it's TensorFlow.

Please refer to MNIST For ML Beginners.

MNIST in Introduction to Private Chainer and Handwriting recognition using KNN in Python However, using this to "make a number discriminator" is the basis of the machine learning tutorial. In other words, it's like a person who is new to the Oracle database enters from the SQL statement that selects the scott / tiger emp database.

From the preparation of TensorFlow

Use TensorFlow from Python. Since the Python environment is created with Anaconda, install it as follows according to Documentation.

% conda install -c conda-forge tensorflow

This completes the installation.

Return to MNIST For ML Beginners

Execute the following code without knowing the reason.

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

For the time being, when I ran it with ipython, it became as follows.

In [1]: from tensorflow.examples.tutorials.mnist import input_data

In [2]: mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting MNIST_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz

Apparently, the data has been downloaded.

% % ls -s1 MNIST_data/
total 22672
 3224 t10k-images-idx3-ubyte.gz
   16 t10k-labels-idx1-ubyte.gz
19368 train-images-idx3-ubyte.gz
   64 train-labels-idx1-ubyte.gz

MNIST implementation in TensorFlow

According to MNIST For ML Beginners, the 28x28 pixel handwritten digit image is either 0-9, as was the chainer tutorial. Is there? Is evaluated by the linear equation of y = Wx + b. The difference is that in the chainer tutorial, what was chainer.links.Linear () should be evaluated using Softmax Regression. I don't understand the difference in this area because I haven't studied enough.

According to Tutorial, the image data is 28x28 pixels per number, but this is one-dimensional 28x28 = 784 array data. It seems that it is assumed that there are 55,000 of them. That is, it is assumed that all the data is a matrix of 784 rows × 55000 columns, and this is mnist.train.xs. The label of each column (that is, which indicates which column is 0 to 9?) Is held as a matrix of 10 rows × 55000 columns, and the sole is mnist.train.ys. Why 10 lines? Speaking of which, 0 is [1,0,0,0,0,0,0,0,0,0] and 3 is [0,0,0,1,0,0,0,0,0,0]. It seems that it is because it is expressed as.

Data reading and formula definition

import tensorflow as tf
x = tf.placeholder(tf.float32, [None, 784])

First, define the data of 784 columns as a placeholder.

W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))

Define the parameters to be learned as Variable.

Then define the model.

y = tf.nn.softmax(tf.matmul(x, W) + b)

matmul () is a matrix multiplication. Here, y can be obtained by softmax regression. Note that y is a 10-row matrix shown as follows.

<tf.Tensor 'Softmax_1:0' shape=(?, 10) dtype=float32>

Preparing for optimization

y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))

By the way, I think that optimization is the least squares method, but apparently it uses the cross_entropy function called -Σy'log (y). So, in order to minimize this cross_entropy, [gradient descent algorithm](https://ja.wikipedia.org/wiki/%E6%9C%80%E6%80%A5%E9%99%8D%E4 % B8% 8B% E6% B3% 95) is used. This is very similar to the SGD we called in the chainer tutorial, but it's a different approach.

train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)

Now, initialize the variables and get ready for execution. The definition of variable initialization, but according to the Tutorial

init = tf.global_variables_initializer()


 It seems that it should be written, but in reality this is an error.

Traceback (most recent call last): File "./mnist_train_tf.py", line 14, in init = tf.global_variables_initializer() AttributeError: module 'tensorflow' has no attribute 'global_variables_initializer'


 Apparently, according to https://github.com/tensorflow/tensorflow/issues/5514#issuecomment-260840208, it seems that `` `global_variables_initializer ()` `` was defined from r0.12 of tensorflow, before r0.11. Seems to have used ```initialize_all_variables ()` ``. The tutorial is r0.12 compliant, but the tensorflow installed in the anaconda environment is r0.11, so it seems that an error occurred.

## Trial for optimization
 Define a Session with tensorflow, and select 100 from the test data in that session to learn.
 That is repeated 1000 times.

```python
sess = tf.Session()
sess.run(init)
for i in range(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

The train_step defined above is for minimizing cross_entropy, and the data batch_xs and label batch_ys obtained from mnist are input to it via the argument feed_dict.

Execution result

20170104_004.png

Since it is 0.9184, I think it is unexpectedly low as a numerical value.

Tutorial also says like this.

This should be about 92%.

Is that good? Well, not really. In fact, it's pretty bad. This is because we're using a very simple model. With some small changes, we can get to 97%. The best models can get to over 99.7% accuracy!

Well, it seems that it's because I used a simple model for the tutorial.

Today's summary

--I tried using TensorFlow --It was different from chainer --There was a method with different names in release 0.12 and 0.11

It turned out that even with the neural network approach, there are completely different implementation methods for each tool. I also found that I didn't have enough knowledge about neural networks and artificial intelligence to understand this code properly.

Today's code


2017/02/22 postscript

With the release of TensorFlow 1.0, the installation method has changed. Try it on your Windows machine.

Installation on Windows

It says to install with pip instead of conda at https://www.tensorflow.org/install/install_windows.

that?

For the time being, when I run the sample code to check if it was installed, it works, but I get a lot of warnings.

>>> import tensorflow as tf
>>> hello = tf.constant('Hello, Tensorflow!')
>>> sess = tf.Session()
>>> print(sess.run(hello))
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "BestSplits" device_type: "CPU"') for unknown op: BestSplits
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "CountExtremelyRandomStats" device_type: "CPU"') for unknown op: Cou
ntExtremelyRandomStats
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "FinishedNodes" device_type: "CPU"') for unknown op: FinishedNodes
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "GrowTree" device_type: "CPU"') for unknown op: GrowTree
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "ReinterpretStringToFloat" device_type: "CPU"') for unknown op: Rein
terpretStringToFloat
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "SampleInputs" device_type: "CPU"') for unknown op: SampleInputs
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "ScatterAddNdim" device_type: "CPU"') for unknown op: ScatterAddNdim

E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "TopNInsert" device_type: "CPU"') for unknown op: TopNInsert
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "TopNRemove" device_type: "CPU"') for unknown op: TopNRemove
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "TreePredictions" device_type: "CPU"') for unknown op: TreePredictio
ns
E c:\tf_jenkins\home\workspace\release-win\device\cpu\os\windows\tensorflow\core\framework\op_ke
rnel.cc:943] OpKernel ('op: "UpdateFertileSlots" device_type: "CPU"') for unknown op: UpdateFert
ileSlots
b'Hello, Tensorflow!'

Hello Tensorflow! Is displayed at the end, but what is this on the way?

Recommended Posts

An introduction to private TensorFlow
Introduction to Private Chainer
An introduction to machine learning
An introduction to Python Programming
An introduction to Bayesian optimization
An introduction to Mercurial for non-engineers
Introduction to TensorFlow --Hello World Edition
An introduction to Python for non-engineers
[Python Tutorial] An Easy Introduction to Python
Introduction to MQTT (Introduction)
Introduction to Scrapy (1)
Introduction to Scrapy (3)
Introduction to Supervisor
Introduction to Tkinter 1: Introduction
An introduction to OpenCV for machine learning
Introduction to PyQt
Introduction to Scrapy (2)
Recurrent Neural Networks: An Introduction to RNN
Probably the most straightforward introduction to TensorFlow
[Linux] Introduction to Linux
Introduction to Scrapy (4)
Beginners read "Introduction to TensorFlow 2.0 for Experts"
Introduction to discord.py (2)
An introduction to Python for machine learning
Introduction to discord.py
An Introduction to Object-Oriented-Give an object a child.
An introduction to Python for C programmers
[What is an algorithm? Introduction to Search Algorithm] ~ Python ~
An introduction to object-oriented programming for beginners by beginners
An introduction to Cython that doesn't go deep
An introduction to statistical modeling for data analysis
[Introduction to TensorBoard] Visualize TensorFlow processing to deepen understanding
Introduction to Python "Re" 1 Building an execution environment
An introduction to voice analysis for music apps
An introduction to Cython that doesn't go deep -2-
Introduction to Lightning pytorch
Introduction to Web Scraping
Introduction to Nonparametric Bayes
Introduction to EV3 / MicroPython
Introduction to Python language
Introduction to TensorFlow-Image Recognition
Introduction to OpenCV (python)-(2)
Introduction to PyQt4 Part 1
Introduction to Dependency Injection
Introduction to machine learning
An introduction to Python distributed parallel processing with Ray
Reading Note: An Introduction to Data Analysis with Python
An introduction to Word2Vec that even cats can understand
[Introduction to RasPi4] Environment construction; OpenCV / Tensorflow, Japanese input ♪
I tried to find an alternating series with tensorflow
[Introduction to TensorBoard: image] TensorFlow Visualize image processing to deepen understanding
An introduction to machine learning from a simple perceptron
AOJ Introduction to Programming Topic # 1, Topic # 2, Topic # 3, Topic # 4
Introduction to electronic paper modules
A quick introduction to pytest-mock
Introduction to dictionary lookup algorithm
[Learning memorandum] Introduction to vim
Introduction to PyTorch (1) Automatic differentiation
opencv-python Introduction to image processing
Introduction to Python Django (2) Win
Kubernetes Scheduler Introduction to Homebrew