Understanding TensorFlow Arithmetic

TensorFlow is primarily a library for implementing machine learning, especially multi-layer neural networks (deep learning), but you don't need to do that hard to understand how it works.

In this article, I would like to unravel the mechanism of TensorFlow based on simple calculations such as arithmetic.

Features of TensorFlow

First, I would like to summarize the features of TensorFlow. As the name implies, TensorFlow is a tool for writing Flow (calculation processing) of Tensor (corresponding to multidimensional arrays, matrices, etc.). Its features are as follows.

Also noteworthy is that it has a track record of being used in various services (search ranking, image classification) within Google.

Please refer to the following part of the official page for details on the above contents.

How TensorFlow works

The mechanism of TensorFlow is as follows.

image

A TensorFlow Graph consists of Node and edge. Node represents the calculation process (green circle in the figure) or the input / output at the end (blue circle in the figure). edge represents a multidimensional array, or tensor, which is the value of the calculation result. Then Graph is assigned to Session to perform the calculation. Session calculates the computable Nodes (Nodes with all the calculation results sent from the edge) asynchronously / in parallel. When calculating, we also allocate which Device (cpu / gpu) to use.

This is the whole picture of the calculation process in TensorFlow. Please refer to the following part of the official page.

Getting Started

In addition, input / output at the end, specifically, reading files and saving calculation results, etc. are summarized in the following places.

All of these are processes that are often used when actually calculating using data, so be sure to check.

Calculation by TensorFlow

Now, I would like to actually use TensorFlow to perform math ... simple operations. The code introduced this time is summarized in the following repository.

icoxfog417/tensorflow-arithmetic

You can refer to the iPython notebook from tensorflow-arithmetic.ipynb.

Installation

See below for installation instructions. It is recommended to use pip install.

Getting Started/Download and Setup

Calculation

Now, let's actually calculate using TensorFlow. At first, I will write the following formula (although math is a little more than math).

y = x^2 + b

The formula written in TensorFlow is as follows.

import tensorflow as tf

def x2_plus_b(x, b):
    _x = tf.constant(x)
    _b = tf.constant(b)
    result = tf.square(_x)
    result = tf.add(result, _b)
    return result

It looks like the figure below.

image

Now that the expression is complete, let's actually execute it using Session.

with tf.Session() as sess:
    result = sess.run([x2_plus_b(2., 3.)])
    print(result)

The execution result (output of print) is as follows.

[7.0]

Since $ 2 ^ 2 + 3 = 7 $, you can see that the calculation is done properly. The methods for the available operations are documented.

TensorFlow API/Math

The argument can also be given in the form placeholder.

import tensorflow as tf

p_x = tf.placeholder(tf.float32)
p_b = tf.placeholder(tf.float32)
p_x2_plus_b = tf.add(tf.square(p_x), p_b)

with tf.Session() as sess:
    result = sess.run([p_x2_plus_b], feed_dict={p_x: [2.], p_b: [3.]})
    print(result)

A value is passed to placeholder using feed_dict when executing Session.

Visualization

One of the features of TensorFlow is that it has a powerful visualization tool called TensorBoard. Let's use this to visualize the calculation result of the formula.

If you installed with pip, you should be able to execute the command tensorboard. Visualization is performed using this, but first we will output the value of the object to be visualized.

import tensorflow as tf

def monitor_calculation(x, b):
    title = "b = {0}".format(b)
    c = x2_plus_b(float(x), float(b))
    s = tf.scalar_summary(title, c)
    m = tf.merge_summary([s])  # if you are using some summaries, merge them
    return m

with tf.Session() as sess:
    writer = tf.train.SummaryWriter("log", graph_def=sess.graph_def)    
    xaxis = range(-10, 12)

    for b in range(3):
        for x in xaxis:
            summary_str = sess.run(monitor_calculation(x, b))
            writer.add_summary(summary_str, x)

The point is to summarize the value calculated by tf.scalar_summary. Write the summary calculated in this way with tf.train.SummaryWriter.

When the above process is executed, the file will be output in the log folder. Then run tensorboard as follows:

tensorboard --logdir=/path/to/log-directory

(The path is definitely an absolute path). Then, I think that the graph will be displayed as shown below (* It takes a long time from the start to the display. If I thought that it would not be displayed for a long time, it might suddenly appear) ..

graph.png

Now you can easily plot the calculated values and so on. At the time of machine learning, it will be possible to grasp the learning situation by plotting errors and accuracy.

Also, in the "GRAPH" tab, you can see a visualization of the constructed formula (= graph).

graph.png

Please refer to the following for techniques for drawing graphs (such as creating and summarizing processes using namespaces).

TensorBoard: Graph Visualization

Also, please refer to the following for how to use TensorBoard.

TensorBoard: Visualizing Learning

For how to export the summary, refer to the mnist sample. There is a description of the process in mnist.py (using scalar_summary etc. here), and in fully_connected_feed.py, the process defined in mnist.py is combined and executed. ..

The above is the basic mechanism of TensorFlow. This content is similar to the content of Getting Started.

I think that there will be more and more articles about machine learning with TensorFlow, so I hope it helps you to understand them.

Recommended Posts

Understanding TensorFlow Arithmetic
Understanding VQ-VAE
Understanding Concatenate
Tensorflow Glossary
tensorflow mnist_deep.py
TensorFlow tutorial tutorial
[Introduction to TensorBoard] Visualize TensorFlow processing to deepen understanding