I tried to learn the angle from sin and cos with chainer

Introduction

Until previous, we learned only 1 input and 1 output from sin like angle (theta), but this time we learned angle from sin and cos I would like to learn 2 inputs and 1 output.

environment

• python: 2.7.6
• chainer: 1.8.0

content of study

Learn the angle (theta) from sin and cos (0-2π).

[training data]

• input: sin, cos (0 ~ 2π, 1000 divisions)
• output: theta

Training data

def get_dataset(N):
    theta = np.linspace(0, 2 * np.pi, N)
    sin = np.sin(theta)
    cos = np.cos(theta)

    x = np.c_[sin, cos]
    y = theta

    return x, y

Implementation

Setting the number of units

Since the number of units in the input layer (in_units) and output layer (out_units) is set, the versatility is slightly increased compared to the previous code.

Batch learning

class MyChain(Chain):
    def __init__(self, in_units=1, n_units=10, out_units=1):
        super(MyChain, self).__init__(
             l1=L.Linear(in_units, n_units),
             l2=L.Linear(n_units, n_units),
             l3=L.Linear(n_units, out_units))
        self.in_units = in_units
        self.out_units = out_units

Learning parameters

• Mini batch size: 20
• Epoch (n_epoch): 500
• Number of hidden layers: 2
• Number of units in the input layer (in_units): 2
• Number of hidden layer units (n_units): 100
• Number of output layer units (out_units): 1
• Activation function: Rectifier (relu)
• Dropout: None (0%)
• Optimization: Adam
• Loss Error Function: Mean Squared Error Function (mean_squared_error)

All parameters are appropriate.

Whole code

The entire

# -*- coding: utf-8 -*-

#Import from one end for the time being
import numpy as np
import chainer
from chainer import cuda, Function, gradient_check, Variable, optimizers, serializers, utils
from chainer import Link, Chain, ChainList
import chainer.functions as F
import chainer.links as L
import time
from matplotlib import pyplot as plt

#data
def get_dataset(N):
    theta = np.linspace(0, 2 * np.pi, N)
    sin = np.sin(theta)
    cos = np.cos(theta)

    x = np.c_[sin, cos]
    y = theta

    return x, y

#neural network
class MyChain(Chain):
    def __init__(self, in_units=1, n_units=10, out_units=1):
        super(MyChain, self).__init__(
             l1=L.Linear(in_units, n_units),
             l2=L.Linear(n_units, n_units),
             l3=L.Linear(n_units, out_units))
        self.in_units = in_units
        self.out_units = out_units

    def __call__(self, x_data, y_data):
        x = Variable(x_data.astype(np.float32).reshape(len(x_data),self.in_units)) #Convert to Variable object
        y = Variable(y_data.astype(np.float32).reshape(len(y_data),self.out_units)) #Convert to Variable object
        return F.mean_squared_error(self.predict(x), y)

    def  predict(self, x):
        h1 = F.relu(self.l1(x))
        h2 = F.relu(self.l2(h1))
        h3 = self.l3(h2)
        return h3

    def get_predata(self, x):
        return self.predict(Variable(x.astype(np.float32).reshape(len(x),self.in_units))).data

# main
if __name__ == "__main__":

    #Training data
    N = 1000
    x_train, y_train = get_dataset(N)

    #test data
    N_test = 900
    x_test, y_test = get_dataset(N_test)

    #Learning parameters
    batchsize = 10
    n_epoch = 500
    in_units = 2
    n_units = 100
    out_units = 1

    #Modeling
    model = MyChain(in_units, n_units, out_units)
    optimizer = optimizers.Adam()
    optimizer.setup(model)

    #Learning loop
    print "start..."
    train_losses =[]
    test_losses =[]
    start_time = time.time()
    for epoch in range(1, n_epoch + 1):

        # training
        perm = np.random.permutation(N)
        sum_loss = 0
        for i in range(0, N, batchsize):
            x_batch = x_train[perm[i:i + batchsize]]
            y_batch = y_train[perm[i:i + batchsize]]
            model.zerograds()
            loss = model(x_batch,y_batch)
            sum_loss += loss.data * batchsize
            loss.backward()
            optimizer.update()

        average_loss = sum_loss / N
        train_losses.append(average_loss)

        # test
        loss = model(x_test, y_test)
        test_losses.append(loss.data)

        #Output learning process
        if epoch % 10 == 0:
            print "epoch: {}/{} train loss: {} test loss: {}".format(epoch, n_epoch, average_loss, loss.data)

    interval = int(time.time() - start_time)
    print "Execution time(normal): {}sec".format(interval)
    print "end"

    #Graphing the error
    plt.plot(train_losses, label = "train_loss")
    plt.plot(test_losses, label = "test_loss")
    plt.yscale('log')
    plt.legend()
    plt.grid(True)
    plt.title("loss")
    plt.xlabel("epoch")
    plt.ylabel("loss")
    plt.show()

    #Graphing learning results
    sin, cos = np.hsplit(x_test ,in_units)
    theta = y_test
    test = model.get_predata(x_test)

    plt.subplot(3, 1, 1)
    plt.plot(theta, sin, "b", label = "sin")
    plt.legend(loc = "upper left")
    plt.grid(True)
    plt.xlim(0, 2 * np.pi)
    plt.ylim(-1.2, 1.2)

    plt.subplot(3, 1, 2)
    plt.plot(theta, cos, "g", label = "cos")
    plt.legend(loc = "upper left")
    plt.grid(True)
    plt.xlim(0, 2 * np.pi)
    plt.ylim(-1.2, 1.2)

    plt.subplot(3, 1, 3)
    plt.plot(theta, theta, "r", label = "theta")
    plt.plot(theta, test, "c", label = "test")
    plt.legend(loc = "upper left")
    plt.grid(True)
    plt.xlim(0, 2 * np.pi)
    plt.ylim(-0.5, 7)

    plt.tight_layout()
    plt.show()

Execution result

error

If the number of epochs is 500, the error is large. Moreover, since the degree of error reduction is becoming saturated, it is unlikely that further reduction can be expected unless the number of epochs is significantly increased.

Learning results

Since the output is linear, the error does not seem to be so large.

When the input is multiplied by the carrier

I also confirmed the case where the input sin and cos were multiplied by the carrier wave. (3 inputs and 1 output including carrier wave) This is the same principle as the resolver that detects the angle of the motor.

Learning data with carrier wave

def get_dataset(N):
    theta = np.linspace(0, 2 * np.pi, N)
    ref = np.sin(40 * theta) #Carrier
    sin = np.sin(theta) * ref
    cos = np.cos(theta) * ref

    x = np.c_[sin, cos, ref]
    y = theta

    return x, y

# in_Change units to 3

It can be confirmed that the error is large even in appearance because the input is complicated. Harmonic components remain.

Summary

Although the error is large, we were able to learn the angle from sin and cos. (2 inputs and 1 output) The angle could also be learned from the waveform of the resolver multiplied by the carrier wave. However, a harmonic component was generated. It was a trivial learning content, so I thought I could learn more accurately, but I was surprised that the error was large. In the future, I want to learn more complicated contents, so I can think of the future ...