I tried to approximate the sin function using chainer

After studying using one of the deep learning frameworks chainer, I tried to approximate the sin function, but it didn't work. Code that just rewrites mnist in the chainer example. I don't know if the dataset is bad or if there is a bug in the learning process. I would be very happy if you could point it out.

First, create a dataset

make_dataset.py


def make_dateset():
	x_train = np.arange(0,3.14*50.0,0.05)
	y_train = np.sin(x_train).astype(np.float32)
	x_test  = np.arange(3.14*50.0,3.14 * 60.0,0.05)
	y_test = np.sin(x_test).astype(np.float32)
	return x_train.astype(np.float32),y_train.astype(np.float32),x_test.astype(np.float32),y_test.astype(np.float32)

x_train contains numbers from 0 to 3.14 * 50 in 0.05 increments in np.array ex) [0.0,0.05,0.10,......,157.0]

y_train contains the value of x_train assigned to the sin function. ex) [0.0,0.47942553860420301,0.09983341......,]

x_test and y_test are similar. Only the range is different

sin_test.py


import numpy as np
import six
import chainer
from chainer import computational_graph as c
from chainer import cuda
import chainer.functions as F
from chainer import optimizers
import matplotlib.pyplot as plt
import csv

def make_dateset():
	x_train = np.arange(0,3.14*50.0,0.05)
	y_train = np.sin(x_train).astype(np.float32)
	x_test  = np.arange(3.14*50.0,3.14 * 60.0,0.05)
	y_test = np.sin(x_test).astype(np.float32)
	return x_train.astype(np.float32),y_train.astype(np.float32),x_test.astype(np.float32),y_test.astype(np.float32)


	
def forward(x_data,y_data,train = True):
	x,t = chainer.Variable(x_data),chainer.Variable(y_data)
	h1 = F.dropout(F.relu(model.l1(x)),  train=train)
	h2 = F.dropout(F.relu(model.l2(h1)),  train=train)
        h3 = F.dropout(F.relu(model.l3(h1)),  train=train)
	y = model.l4(h3)
	return F.mean_squared_error(y,t),y


if __name__ == "__main__":
	
	x_train,y_train,x_test,y_test = make_dateset()
	x_train,y_train = x_train.reshape(len(x_train),1),y_train.reshape(len(y_train),1)
	x_test,y_test = x_test.reshape(len(x_test),1),y_test.reshape(len(y_test),1)
	y_t,y_p,ll = [],[],[]
	
	xp = np

	batchsize = 10
	N = len(x_train)
	N_test = len(x_test)
	n_epoch = 100
	n_units = 20
	pred_y = []

	model = chainer.FunctionSet(l1=F.Linear(1, n_units),
								l2=F.Linear(n_units, n_units),
								l3=F.Linear(n_units, u_units)),
                                                                l4=F.Linear(n_units, 1))
	optimizer = optimizers.Adam()
	optimizer.setup(model.collect_parameters())



	x_t,y_t,y_p = [],[],[]

	for epoch in six.moves.range(1, n_epoch + 1):
		print('epoch', epoch)

		perm = np.random.permutation(N)
		sum_loss = 0

		for i in six.moves.range(0, N, batchsize):
			x_batch = xp.asarray(x_train[perm[i:i + batchsize]])
			y_batch = xp.asarray(y_train[perm[i:i + batchsize]])

			optimizer.zero_grads()
			loss,y = forward(x_batch, y_batch)
			loss.backward()
			optimizer.update()
			sum_loss += float(cuda.to_cpu(loss.data)) * len(y_batch)
		print "train mean loss = ",sum_loss/N 
		
		sum_loss = 0
		for i in six.moves.range(0, N_test, batchsize):
			x_batch = xp.asarray(x_test[i:i + batchsize])
			y_batch = xp.asarray(y_test[i:i + batchsize])
			loss, y = forward(x_batch, y_batch, train=False)
                        #For debugging
			#y_t.append(y_batch[0])
			#y_p.append(y.data[0])
			#x_t.append(x_batch[0])
			sum_loss += float(cuda.to_cpu(loss.data)) * len(y_batch)
		print "test mean loss is ",sum_loss/N_test
    #For debugging
	#f = open('sin_pre.csv','ab')
	#csvWriter = csv.writer(f)
	#csvWriter.writerow(y_p)
	#f.close()

	#f = open('sin_ans.csv','ab')
	#csvWriter = csv.writer(f)
	#csvWriter.writerow(y_t)
	#f.close()

The first x_train, y_train, x_test, y_test are converted to N * 1 matrix. (To be able to pass it to Chainer's Variable) The rest is almost the same as the mnist sample. The difference is the regression problem, so the Forward () function uses mean_squared_error (mean squared error function) to find the loss. After that, the network configuration is 1-20-20-1.

Execution result(Up to epoch10)


('epoch', 1)
train mean loss =  2553.66754833
test mean loss is  127.272548827
('epoch', 2)
train mean loss =  401.413729346
test mean loss is  5.86524515122
('epoch', 3)
train mean loss =  138.270190761
test mean loss is  4.34996299998
('epoch', 4)
train mean loss =  68.4881465446
test mean loss is  0.659433874475
('epoch', 5)
train mean loss =  38.2469408746
test mean loss is  0.640729590383
('epoch', 6)
train mean loss =  24.6955423482
test mean loss is  0.529370371471
('epoch', 7)
train mean loss =  16.3685227446
test mean loss is  0.505678843091
('epoch', 8)
train mean loss =  11.0349840385
test mean loss is  0.542997811425
('epoch', 9)
train mean loss =  7.98288726631
test mean loss is  0.509733980175
('epoch', 10)
train mean loss =  5.89249175341
test mean loss is  0.502585373718

It seems that I am learning, but the test mean loss is wandering around 0.5 from around epoch exceeding 20, and there is no sign that it will drop further. I don't know if I've fallen into a local solution, the parameters haven't been updated, or simply a mistake in the code.

Reference chainer

Recommended Posts

I tried to approximate the sin function using chainer
I tried to approximate the sin function using chainer (re-challenge)
I tried to learn the sin function with chainer
I tried to get the index of the list using the enumerate function
I tried to learn the angle from sin and cos with chainer
I tried to identify the language using CNN + Melspectogram
I tried to complement the knowledge graph using OpenKE
I tried to compress the image using machine learning
I tried to move the ball
I tried using the checkio API
I tried to estimate the interval.
I tried to simulate ad optimization using the bandit algorithm.
I tried to implement the mail sending function in Python
[TF] I tried to visualize the learning result using Tensorboard
I tried to fight the Local Minimum of Goldstein-Price Function
I tried to output the access log to the server using Node.js
I tried using Azure Speech to Text.
I tried to summarize the umask command
I tried to recognize the wake word
I tried to classify text using TensorFlow
I tried to summarize the graphical modeling.
I tried to digitize the stamp stamped on paper using OpenCV
I tried to estimate the pi stochastically
I tried to touch the COTOHA API
I tried using the BigQuery Storage API
I tried to predict Covid-19 using Darts
I tried to transform the face image using sparse_image_warp of TensorFlow Addons
I tried to execute SQL from the local environment using Looker SDK
I tried to get the batting results of Hachinai using image processing
I tried to estimate the similarity of the question intent using gensim's Doc2Vec
765 I tried to identify the three professional families by CNN (with Chainer 2.0.0)
I tried to control multiple servo motors MG996R using the servo driver PCA9685.
I tried to summarize various sentences using the automatic summarization API "summpy"
I tried to extract and illustrate the stage of the story using COTOHA
I tried the common story of using Deep Learning to predict the Nikkei 225
Using COTOHA, I tried to follow the emotional course of Run, Melos!
I tried to analyze the New Year's card by myself using python
I tried web scraping to analyze the lyrics.
I tried the pivot table function of pandas
[Python] I tried substituting the function name for the function name
I tried to optimize while drying the laundry
I tried to save the data with discord
I tried to synthesize WAV files using Pydub.
I tried using PyCaret at the fastest speed
I tried using the Google Cloud Vision API
I tried to touch the API of ebay
I tried to correct the keystone of the image
I tried using the Datetime module by Python
Qiita Job I tried to analyze the job offer
I tried using the image filter of OpenCV
LeetCode I tried to summarize the simple ones
I tried using the functional programming library toolz
I want to use the activation function Mish
I tried to implement the traveling salesman problem
I tried to make a ○ ✕ game using TensorFlow
I tried to predict the price of ETF
I tried to vectorize the lyrics of Hinatazaka46!
I tried using parameterized
I tried using argparse
I tried using mimesis
I tried using anytree