I will explain a program that approximates sin functions while also studying how to use TensorFlow.
About X (input value) and t (teacher data). Save training set data in Placeholder For example, [None, 5] represents a matrix of [arbitrary x 5]. t is set to [None, 1] so that the number of data can be taken arbitrarily.
About w (weight). Since it corresponds to the Variable to be optimized, it is defined as an instance of the tf.Variable class. Make a 5x1 matrix with an initial value of 0.
These create a network of input layer 1, intermediate layer 5, and output layer 1.
x=tf.placeholder(tf.float32,[None,5])
w=tf.Variable(tf.zeros([5,1]))
t=tf.placeholder(tf.float32,[None,1])
The expression of y = Xw is expressed as follows.
y=tf.matmul(x,w)
tf.matmul is a matrix multiplication function that uses the placeholder x and Variable w defined earlier. No specific value has been entered yet, and y has not been determined.
The difference between the teacher data t and the output y is the sum of squares. We will learn to minimize this squared error. Also select the optimization algorithm at this time.
loss=tf.reduce_sum(tf.square(y-t))
train_step=tf.train.AdamOptimizer().minimize(loss)
Variables corresponding to Variable are calculated using the "session" that is the execution environment of the training algorithm. First, prepare a new session and initialize Variable.
sess=tf.Session()
sess.run(tf.initialize_all_variables())
Substitute the training set data into the Placeholder. train_t is the actual data.
train_t=np.array([np.sin(2.0*np.pi*i/11) for i in range(12)])
train_t=train_t.reshape([12,1])
train_x=np.zeros([12,5])
for row,m in enumerate(range(1,13)):
for col,n in enumerate(range(0,5)):
train_x[row][col]=m**n
Execute the defined training algorithm train_step to correct the weights. At this time, feed_dict sets a specific value for Placeholder. You can also evaluate loss within a session and retrieve the current value, such as loss_val. Then, it outputs the value of the parameter at the time after the training is stopped (after exiting the for loop).
i=0
for _ in range(100000):
i+=1
sess.run(train_step,feed_dict={x:train_x,t:train_t})
if i % 1000==0:
loss_val=sess.run(loss,feed_dict={x:train_x,t:train_t})
print ('Step:%d, Loss:%f'%(i,loss_val))
w_val=sess.run(w)
print w_val
The curve of the test result is output by the following formula. y(x)=w0+w1x+w2x^2+w3x^3+w4x^4
def predict(x):
result=0.0
for n in range(0,5):
result+=w_val[n][0]* x**n
return result
When the middle layer is 5 layers and the number of learning is 10,000 times Although it deviates from the plot, I think that it can be approximated.
Try changing the number of intermediate layers ... Intermediate layer 6 layers: It has shifted more.
Intermediate layer 4 layers: It is closer than 5 layers.
It is also important to specify the appropriate number of intermediate layers (hyperparameters). ..
sin.py
#-*- coding:utf-8 -*-
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
x=tf.placeholder(tf.float32,[None,5])
w=tf.Variable(tf.zeros([5,1]))
y=tf.matmul(x,w)
t=tf.placeholder(tf.float32,[None,1])
loss=tf.reduce_sum(tf.square(y-t))
train_step=tf.train.AdamOptimizer().minimize(loss)
sess=tf.Session()
sess.run(tf.initialize_all_variables())
train_t=np.array([np.sin(2.0*np.pi*i/11) for i in range(12)])
train_t=train_t.reshape([12,1])
train_x=np.zeros([12,5])
for row,m in enumerate(range(1,13)):
for col,n in enumerate(range(0,5)):
train_x[row][col]=m**n
i=0
for _ in range(100000):
i+=1
sess.run(train_step,feed_dict={x:train_x,t:train_t})
if i % 1000==0:
loss_val=sess.run(loss,feed_dict={x:train_x,t:train_t})
print ('Step:%d, Loss:%f'%(i,loss_val))
w_val=sess.run(w)
print w_val
def predict(x):
result=0.0
for n in range(0,5):
result+=w_val[n][0]* x**n
return result
fig=plt.figure()
subplot=fig.add_subplot(1,1,1)
subplot.set_xlim(1,12)
subplot.scatter(range(1,13),train_t)
linex=np.linspace(1,12,100)
liney=predict(linex)
subplot.plot(linex,liney)
plt.show()
Recommended Posts