Summary of various operations in Tensorflow

Introduction

Since Tensorflow handles a quantity called a tensor, Numpy methods etc. cannot be used. Therefore, it is necessary to use the method provided by Tensorflow.

Here is a summary of the basic four arithmetic operations and matrix operations. In particular, it is compared with various operations defined in Python (+, -,/,//,% ).

Caution --Not all operations are summarized, so please refer to Official Documents for other operations. --Since Tensorflow handles all tensors, when referring to the type of data, we skip the tensors and write "list is ~", "vector is ~", etc. I would like you to pay attention to that point. --2018/01/31: Tensorflow v1.5.0 released was released on January 26th. From this version, a method called eager execution has been implemented, and ** Sesssion.run () You can now do ** calculations without having to do it. Please refer to the note about this added at the end of the article.

Operating environment

Tensorflow 1.4.1

Four arithmetic operations

Addition tf.add

import tensorflow as tf

const1 = tf.constant(2)
const2 = tf.constant(3)
add_op = tf.add(const1, const2) # 2 + 3
with tf.Session() as sess:
    result = sess.run(add_op)
    print(result)
5

Corresponds to Python 3 addition +.

Subtraction tf.subtract

import tensorflow as tf

const1 = tf.constant(2)
const2 = tf.constant(3)
subt_op = tf.subtract(const1, const2) # 2 - 3

with tf.Session() as sess:
    result = sess.run(subt_op)
    print(result)
-1

This is the same as -.

Division --Python2 ver -- tf.div


import tensorflow as tf

const1 = tf.constant(2)
const2 = tf.constant(3)
div2_op = tf.div(const1, const2) # 2 / 3 in Python2

with tf.Session() as sess:
    result = sess.run(div_op)
    print(result)
0

Corresponds to division / in Python2. Official recommends the following tf.divide.

Division --Python3 ver -- tf.divide

import tensorflow as tf

const1 = tf.constant(2)
const2 = tf.constant(3)
div_op = tf.divide(const1, const2) # 2 / 3 in Python3

with tf.Session() as sess:
    result = sess.run(div_op)
    print(result)
0.666666666667

Equivalent to / in Python 3

Integer division tf.floordiv

import tensorflow as tf

const1 = tf.constant(2)
const2 = tf.constant(3)
fdiv_op = tf.floordiv(const1, const2) # 2 // 3

with tf.Session() as sess:
    result = sess.run(fdiv_op)
    print(result)
0

Corresponds to the // of Python3. (Truncate after the decimal point)

Modulo operation tf.mod

import tensorflow as tf

const1 = tf.constant(2)
const2 = tf.constant(4)
mod_op = tf.mod(const1, const2) # 2 % 3 

with tf.Session() as sess:
    result = sess.run(mod_op)
    print(result)
2

Corresponds to the % of Python3.

Scalar product tf.scalar_mul

import tensorflow as tf

const1 = tf.constant(2) #Declare a constant(Value is 2)
const2 = tf.constant(3)
smul_op = tf.scalar_mul(const1, const2) # 2 * 3 

with tf.Session() as sess:
    result = sess.run(smul_op)
    print(result)
6

The product of constants.

Vector cross product tf.cross

import tensorflow as tf

const1 = tf.constant([1, 2, 3])
const2 = tf.constant([4, 5, 6])
cross_op = tf.cross(const1, const2) # [1, 2, 3] x [4, 5, 6]

with tf.Session() as sess:
    result = sess.run(cross_op)
    print(result)
[-3  6 -3]

As for the vector product (outer product), the definition will come out if you google, so I will omit it here.

Matrix operation

Matrix with all 1 elements tf.ones


import numpy as np
import tensorflow as tf

t = tf.ones([3, 2])

with tf.Session() as sess:
    print("tf.ones([3, 2]) = %s" % sess.run(t))    
tf.ones([3, 2]) = [[ 1.  1.]
 [ 1.  1.]
 [ 1.  1.]]

tf.ones ([3,2]): Create a 3x2 matrix with all elements 1

Matrix with all 0 elements tf.zeros


import numpy as np
import tensorflow as tf

t = tf.zeros([5])

with tf.Session() as sess:
    print("tf.zeros([5]) = %s" % sess.run(t))
tf.zeros([5]) = [ 0.  0.  0.  0.  0.]

tf.zeros ([5]): Create a 1x5 matrix with all zeros (vector with 5 elements)

Matrix with random numbers as elements tf.random_uniform


import numpy as np
import tensorflow as tf

t = tf.random_uniform([1, 3])

with tf.Session() as sess:
    print("tf.random_uniform([1, 3]) = %s" % sess.run(t))
tf.random_uniform([1, 3]) = [[ 0.16699052  0.04628575  0.38685966]]

tf.random_uniform ([1,3]): Create a 1x3 matrix (vector with 3 elements) with randomly generated numbers from 0 to 1.

Outputs the number of elements in the specified interval as a list tf.linspace


import numpy as np
import tensorflow as tf

t = tf.linspace(1.0, 7.0, 4)

with tf.Session() as sess:
    print("tf.linspace(1.0, 7.0, 4) = %s" % sess.run(t))
tf.linspace(1.0, 7.0, 4) = [ 1.  3.  5.  7.]

tf.linspace (1.0,7.0,4): Divide 1.0 to 4.0 into four, generate numbers in ascending order, and output as a list (vector).

Matrix product

import tensorflow as tf

const1 = tf.random_uniform([3,2])
const2 = tf.random_uniform([2,3])
mul_op = tf.matmul(const1, const2)

with tf.Session() as sess:
    print("const1: %s" % sess.run(const1))
    print("const2: %s" % sess.run(const2))
    print("tf.matmul(const1, const2) = %s" % sess.run(mul_op))
const1: [[ 0.22310185  0.76020801]
 [ 0.04765964  0.81115341]
 [ 0.15238702  0.24294829]]
const2: [[ 0.89871132  0.3882308   0.54530931]
 [ 0.60662949  0.27270687  0.20178115]]
tf.matmul(const1, const2) = [[ 0.55778277  0.5268842   0.72335124]
 [ 0.42495638  0.3350077   0.66840851]
 [ 0.45947441  0.2069075   0.99706292]]

Let the matrix to be considered be $ A_ {ij} $, $ B_ {ij} $. Let A be the NxM matrix and B be the KxL matrix. At this time, to perform the product,

--M = K for tf.matmul (A, B) to be possible --L = N for tf.matmul (B, A) to be possible

Must hold. (That is, the number of elements in the column of the first matrix and the number of elements in the row of the second matrix must match.) Note that if this is neglected, errors will occur frequently. In particular, you can use tf.transpose to transpose the matrix, or set the transpose to True with the option of tf.matmul. Please refer to Official Document for this.

Convert Numpy list to tensor tf.convert_to_tensor


import numpy as np
import tensorflow as tf

t = tf.convert_to_tensor(np.linspace(1, 7, 4))

with tf.Session() as sess:
    print("tf.convert_to_tensor( np.linspace(1, 7, 4) ) = %s" % sess.run(t))
tf.convert_to_tensor( np.linspace(1, 7, 4) ) = [ 1.  3.  5.  7.]

tf.convert_to_tensor: Converts the generated Numpy list to a tensor.

Other

Value assignment


import tensorflow as tf

# see https://www.tensorflow.org/api_guides/python/state_ops
print("\n variable\n=====================")

w = tf.Variable(tf.zeros([3, 2])) #variable(queue)Declaration of w. 初期値はゼロqueue

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    
    print("w = %s" % sess.run(w))
    sess.run(tf.assign(w, tf.ones([3, 2])))

    wn = sess.run(w)
    print("w = %s" % wn)
variable
=====================
w = [[ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]]
w = [[ 1.  1.]
 [ 1.  1.]
 [ 1.  1.]]

-- tf.assign (w, tf.ones ([3, 2]): Assign the variable w totf.ones ([3,2])(3x2 matrix with all 1 elements.

This operation is not limited to matrices. (Also possible for scalar variables)

Symbolic operation


import tensorflow as tf

print("\n symbolic variables\n=====================")

a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
a_times_b1 = tf.multiply(a, b) #  multiplication of a and b
a_times_b2 = a * b

with tf.Session() as sess:
    # where "name" is the python symbol
    print("%f should equal 2.0" % sess.run(a_times_b1, feed_dict={a: 1, b: 2}))
    print("%f should equal 9.0" % sess.run(a_times_b1, feed_dict={a: 3, b: 3}))
    print("%f should equal 9.0" % sess.run(a_times_b2, feed_dict={a: 3, b: 3}))
Symbolic variable
=====================
2.000000 should equal 2.0
9.000000 should equal 9.0
9.000000 should equal 9.0

Instead of fixing only one value, substitute several values later using feed_dict.

Reference: Tensorflow concept learned from potato manufacturing

Various functions


import tensorflow as tf

print("\n math function\n=====================")

x = tf.linspace(0., 4., 5)

with tf.Session() as sess:
    print("x = %s" % sess.run(x))
    print("(x+1)**2 - 2) = %s" % sess.run( (x+1)**2 - 2))
    print("sin(x) = %s" % sess.run( tf.sin(x) ))
    print("sum(x) = %s" % sess.run( tf.reduce_sum(x) ))
Mathematical function
=====================
x = [ 0.  1.  2.  3.  4.]
(x+1)**2 - 2) = [ -1.   2.   7.  14.  23.]
sin(x) = [ 0.          0.84147096  0.90929741  0.14112    -0.7568025 ]
sum(x) = 10.0

See official documentation for details

Take a list as a variable

These functions can be listed as variables:

print("\Vector to n argument(sequence,list)Can be taken.(Numpy-like)\n=====================")
tf.sin(3.)
tf.sin([1., 2., 3.])
tf.sin(tf.linspace(0., 10., 20))
tf.sin(np.linspace(0, 10, 20))  # equivalent
tf.sin(tf.ones(shape=(2, 3, 4)))  # 2x3x4 tensor

# Operators (+, -, /, *)
a = tf.zeros(shape=(2, 3))
b = tf.ones(shape=(2, 3))
c = tf.ones(shape=(3, 2))

with tf.Session() as sess:
    print('a + b: %s'% sess.run(a + b))  # same as tf.add(a, b)
    print('a - b: %s' % sess.run(a - b))  # same as tf.subtract(a, b)
    print('a * b: %s' % sess.run(a * b))  # same as tf.mul(a, b)
    print('a / b: %s' % sess.run(a / b))  # same as tf.division(a, b)
# a + c  # doesn't work; tensors need to be of same shape
Vector as an argument(sequence,list)Can be taken.(Numpy-like)
=====================
a + b: [[ 1.  1.  1.]
 [ 1.  1.  1.]]
a - b: [[-1. -1. -1.]
 [-1. -1. -1.]]
a * b: [[ 0.  0.  0.]
 [ 0.  0.  0.]]
a / b: [[ 0.  0.  0.]
 [ 0.  0.  0.]]

About Eager execution

Let's give an example of Eager execution by taking some of the contents dealt with above as an example.

Example 1: Addition

When I rewrite the code of addition,


import tensorflow as tf
import tensorflow.contrib.eager as tfe

tfe.enable_eager_execution()

const1 = tf.constant(2)
const2 = tf.constant(3)
add_op = tf.add(const1, const2) # 2 + 3
print(add_op)
# tf.Tensor(5, shape=(), dtype=int32)

It will be. First of all, be sure to write tfe.enable_eager_execution () at the beginning of the program. (When running on Jupyter, if there is memory left in the kernel

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-2-a35e1dbe978d> in <module>()
      2 import tensorflow.contrib.eager as tfe
      3 
----> 4 tfe.enable_eager_execution()
      5 
      6 const1 = tf.constant(2)

/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in enable_eager_execution(config, device_policy)
   4892   else:
   4893     raise ValueError(
-> 4894         "tfe.enable_eager_execution has to be called at program startup.")
   4895 
   4896 

ValueError: tfe.enable_eager_execution has to be called at program startup.

(Example), so a kernel reboot is essential. By the way, the type of output result is

type(add_op)
# EagerTensor

It has become. If you try adding 3 to this value,


add_op + 3
# <tf.Tensor: id=5, shape=(), dtype=int32, numpy=8>

And it is calculated correctly as the value of numpy. Use the numpy () method to retrieve this value. In other words

add_op2 = add_op + 3
print(add_op2.numpy())
# 8

You can retrieve the value by. The type of this value is

type(add_op2)
# numpy.int32

Since it is, normal four arithmetic operations are possible.

Example 2: Value assignment

In the same way, if you rewrite the value assignment code


import tensorflow as tf
import tensorflow.contrib.eager as tfe

tfe.enable_eager_execution()

# see https://www.tensorflow.org/api_guides/python/state_ops
print("\n variable\n=====================")

w = tfe.Variable(tf.zeros([3, 2])) #variable(queue)Declaration of w. 初期値はゼロqueue

tf.global_variables_initializer()

print("w = %s" % w)
tf.assign(w, tf.ones([3, 2]))

wn = w
print("w = %s" % wn)

#variable
# =====================
# w = <tf.Variable 'Variable:0' shape=(3, 2) dtype=float32, numpy=
# array([[0., 0.],
#        [0., 0.],
#        [0., 0.]], dtype=float32)>
# w = <tf.Variable 'Variable:0' shape=(3, 2) dtype=float32, numpy=
# array([[1., 1.],
#        [1., 1.],
#        [1., 1.]], dtype=float32)>

It will be. Note that we use tfe.Variable when declaring variables here. (As an aside, tf.Placeholder cannot be used when using eager execution.) Other than that, it can be rewritten in the same way by removing all with statements. As before to retrieve the value

print(w.numpy()

# array([[1., 1.],
#       [1., 1.],
#       [1., 1.]], dtype=float32)

You can retrieve the value by doing (as well as wn). As we did in Example 1 above, we can use the numpy () method to retrieve it as a Numpy arrayt and perform Numpy's four arithmetic operations.

Recommended Posts

Summary of various operations in Tensorflow
Summary of various for statements in Python
Summary of Tensorflow / Keras
Summary of string operations
Summary of Excel operations using OpenPyXL in Python
Summary of python file operations
Summary of Python3 list operations
Understand the images of various matrix operations used in Keras (Tensorflow) with examples
Summary of methods often used in pandas
Summary of frequently used commands in matplotlib
Summary of stumbling blocks in installing CaboCha
Summary of modules and classes in Python-TensorFlow2-
Summary of built-in methods in Python list
Summary of operations often performed with asyncpg
Summary of processes often performed in Pandas 1 (CSV, Excel file related operations)
Summary of OSS tools and libraries created in 2016
Summary of how to import files in Python 3
Summary of what was used in 100 Pandas knocks (# 1 ~ # 32)
Summary of tools used in Command Line vol.8
Summary of tools used in Command Line vol.5
Summary of evaluation functions used in machine learning
[Python] Summary of S3 file operations with boto3
Checkpoint format of tf.train.Saver changed in TensorFlow v0.12
Summary of error handling methods when installing TensorFlow (2)
Numerical summary of data
File operations in Python
Numerical representation of days of the week in various languages
Summary of tools needed to analyze data in Python
Various character string operations
Summary of Linux (UNIX) commands that appeared in Progate
Summary of pyenv usage
Old openssl causes problems in various parts of python
File operations in Python
HMAC in various languages
Execution procedure of TensorFlow in batch processing of supercomputer ITO
Separation summary of development environment by chroot of various Linux
Summary of Python arguments
A collection of Excel operations often used in Python
Summary of logrotate software logrotate
Summary of test method
Summary of date processing in Python (datetime and dateutil)
Image normalization in TensorFlow
Various processing of Python
Summary of Prototype patterns introductory design patterns learned in Java language
Basic summary of data manipulation in Python Pandas-Second half: Data aggregation
Summary of Singleton patterns introductory design patterns learned in Java language
[For beginners] Summary of standard input in Python (with explanation)
Summary of stumbling blocks in Django for the first time
I measured various methods of interprocess communication in multiprocessing of python3
Chaos regression of logistic map by petit RNN in Tensorflow
Summary of how to write .proto files used in gRPC
The story of downgrading the version of tensorflow in the demo of Mask R-CNN.
Four arithmetic operations in python
Clipping and normalization in TensorFlow
What's new in Python 3.10 (Summary)
2017.3.6 ~ 3.12 Summary of what we did
Tuning experiment of Tensorflow data
Partial in case of trouble
Various Python built-in string operations
Various import methods of Mnist
List of nodes in diagrams