I tried to learn the sin function with chainer

Introduction

Deep learning beginners tried to learn sin function with chainer. As a beginner level, I felt a little understood after reading deep learning, but when I decided to write this article, I felt despair at my low level of understanding. Learning sin functions has been done by yuukiclass and many others, but it's not bad.

environment

content of study

Learn sin (theta) from angles (theta) from 0 to 2π

[training data]

Implementation

Mini batch learning

This is the implementation part of mini-batch learning. This code is familiar from MNIST samples (some changes such as range). This mini-batch learning seems to be popular.

Excerpt from mini-batch learning


perm = np.random.permutation(N)
sum_loss = 0
for i in range(0, N, batchsize):
    x_batch = x_train[perm[i:i + batchsize]]
    y_batch = y_train[perm[i:i + batchsize]]

    model.zerograds()
    loss = model(x_batch,y_batch)
    sum_loss += loss.data * batchsize
    loss.backward()
    optimizer.update()

Training data & test

The number of data is changed so that the angle used during the test is different from that during learning. The angle is changed by dividing 0 to 2π into 1,000 during learning and dividing 0 to 2π into 900 during testing.

Training data&Excerpt from result confirmation


#Training data
N = 1000
x_train, y_train = get_dataset(N)

#test data
N_test = 900
x_test, y_test = get_dataset(N_test)

'''
abridgement
'''
    # test
    loss = model(x_test,y_test)
    test_losses.append(loss.data)

Learning parameters

All parameters are appropriate.

Whole code

The entire


# -*- coding: utf-8 -*-

#Import from one end for the time being
import numpy as np
import chainer
from chainer import cuda, Function, gradient_check, Variable, optimizers, serializers, utils
from chainer import Link, Chain, ChainList
import chainer.functions as F
import chainer.links as L
import time
from matplotlib import pyplot as plt

#data
def get_dataset(N):
    x = np.linspace(0, 2 * np.pi, N)
    y = np.sin(x)
    return x, y

#neural network
class MyChain(Chain):
    def __init__(self, n_units=10):
        super(MyChain, self).__init__(
             l1=L.Linear(1, n_units),
             l2=L.Linear(n_units, n_units),
             l3=L.Linear(n_units, 1))

    def __call__(self, x_data, y_data):
        x = Variable(x_data.astype(np.float32).reshape(len(x_data),1)) #Convert to Variable object
        y = Variable(y_data.astype(np.float32).reshape(len(y_data),1)) #Convert to Variable object
        return F.mean_squared_error(self.predict(x), y)

    def  predict(self, x):
        h1 = F.relu(self.l1(x))
        h2 = F.relu(self.l2(h1))
        h3 = self.l3(h2)
        return h3

    def get_predata(self, x):
        return self.predict(Variable(x.astype(np.float32).reshape(len(x),1))).data

# main
if __name__ == "__main__":

    #Training data
    N = 1000
    x_train, y_train = get_dataset(N)

    #test data
    N_test = 900
    x_test, y_test = get_dataset(N_test)

    #Learning parameters
    batchsize = 10
    n_epoch = 500
    n_units = 100

    #Modeling
    model = MyChain(n_units)
    optimizer = optimizers.Adam()
    optimizer.setup(model)

    #Learning loop
    train_losses =[]
    test_losses =[]
    print "start..."
    start_time = time.time()
    for epoch in range(1, n_epoch + 1):

        # training
        perm = np.random.permutation(N)
        sum_loss = 0
        for i in range(0, N, batchsize):
            x_batch = x_train[perm[i:i + batchsize]]
            y_batch = y_train[perm[i:i + batchsize]]

            model.zerograds()
            loss = model(x_batch,y_batch)
            sum_loss += loss.data * batchsize
            loss.backward()
            optimizer.update()

        average_loss = sum_loss / N
        train_losses.append(average_loss)

        # test
        loss = model(x_test,y_test)
        test_losses.append(loss.data)

        #Output learning process
        if epoch % 10 == 0:
            print "epoch: {}/{} train loss: {} test loss: {}".format(epoch, n_epoch, average_loss, loss.data)

        #Graphing learning results
        if epoch in [10, 500]:
            theta = np.linspace(0, 2 * np.pi, N_test)
            sin = np.sin(theta)
            test = model.get_predata(theta)
            plt.plot(theta, sin, label = "sin")
            plt.plot(theta, test, label = "test")
            plt.legend()
            plt.grid(True)
            plt.xlim(0, 2 * np.pi)
            plt.ylim(-1.2, 1.2)
            plt.title("sin")
            plt.xlabel("theta")
            plt.ylabel("amp")
            plt.savefig("fig/fig_sin_epoch{}.png ".format(epoch)) #Assuming the fig folder exists
            plt.clf()

    print "end"

    interval = int(time.time() - start_time)
    print "Execution time: {}sec".format(interval)


    #Error graphing
    plt.plot(train_losses, label = "train_loss")
    plt.plot(test_losses, label = "test_loss")
    plt.yscale('log')
    plt.legend()
    plt.grid(True)
    plt.title("loss")
    plt.xlabel("epoch")
    plt.ylabel("loss")
    plt.savefig("fig/fig_loss.png ") #Assuming the fig folder exists
    plt.clf()

Execution result

error

The error tends to decrease as the epoch (number of learnings) increases. There was no significant difference between the learning and testing errors. I think that the error at the time of testing is slightly better than that at the time of learning because the method of calculating the error is different.

fig_loss.png

Learning results

When the epoch is 10, it is hard to say that it is a sin function, but when learning progresses to 500, it is quite close to the sin function.

epoch: 10 fig_sin_epoch10.png

epoch: 500 fig_sin_epoch500.png

Summary

For the time being, I was able to train the sin function with chainer.

However, for some reason, the larger the angle, the larger the error. I thought that if the order of the angles to be learned was randomized, the variation in the error for each angle could be suppressed, but it seems to be different. It is not well understood.

reference

I tried to approximate the sin function using chainer (re-challenge)

Chainer and deep learning learned by function approximation

Regression forward propagation neural network with chainer

Recommended Posts

I tried to learn the sin function with chainer
I tried to approximate the sin function using chainer
I tried to learn the angle from sin and cos with chainer
I tried to approximate the sin function using chainer (re-challenge)
I tried to learn logical operations with TF Learn
I tried to save the data with discord
I tried to learn PredNet
765 I tried to identify the three professional families by CNN (with Chainer 2.0.0)
I tried to implement and learn DCGAN with PyTorch
I tried to touch the CSV file with Python
I tried to solve the soma cube with python
I tried to solve the problem with Python Vol.1
I tried to move the ball
I tried to estimate the interval.
I tried to find the entropy of the image with python
I tried to simulate how the infection spreads with Python
I tried to analyze the whole novel "Weathering with You" ☔️
I tried to find the average of the sequence with TensorFlow
I tried to notify the train delay information with LINE Notify
I tried to implement ListNet of rank learning with Chainer
I tried to implement the mail sending function in Python
I tried to fight the Local Minimum of Goldstein-Price Function
I tried to divide the file into folders with Python
I tried to describe the traffic in real time with WebSocket
I tried to solve the ant book beginner's edition with python
I tried to implement Autoencoder with TensorFlow
Learn to colorize monochrome images with Chainer
I tried to summarize the umask command
I tried to get the index of the list using the enumerate function
I tried to automate the watering of the planter with Raspberry Pi
I tried to visualize AutoEncoder with TensorFlow
I tried to recognize the wake word
I tried to get started with Hy
I tried to process the image in "sketch style" with OpenCV
I tried to summarize the graphical modeling.
I tried to get started with Bitcoin Systre on the weekend
I tried to estimate the pi stochastically
I tried to touch the COTOHA API
I tried to implement CVAE with PyTorch
I tried to predict by letting RNN learn the sine wave
I tried to process the image in "pencil style" with OpenCV
I tried to expand the size of the logical volume with LVM
I tried playing with the image with Pillow
I also tried to imitate the function monad and State monad with a generator in Python
I tried to solve TSP with QAOA
I tried running the DNN part of OpenPose with Chainer CPU
I tried to improve the efficiency of daily work with Python
I tried to make an image similarity function with Python + OpenCV
[Python] I tried to visualize the night on the Galactic Railroad with WordCloud!
I tried to refer to the fun rock-paper-scissors poi for beginners with Python
I tried to express sadness and joy with the stable marriage problem.
I tried to get the authentication code of Qiita API with Python.
I tried to automatically extract the movements of PES players with software
I tried to create CSV upload, data processing, download function with Django
I tried with the top 100 PyPI packages> I tried to graph the packages installed on Python
I tried to analyze the negativeness of Nono Morikubo. [Compare with Posipa]
I tried to streamline the standard role of new employees with Python
I tried to visualize the text of the novel "Weathering with You" with WordCloud
I tried to visualize the model with the low-code machine learning library "PyCaret"
I tried to get the movie information of TMDb API with Python
I tried to predict the behavior of the new coronavirus with the SEIR model.