Various methods to numerically create the inverse function of a certain function Part 1 Polynomial regression

Confirmation of problem

Involute function f(\alpha) = tan\ \alpha - \alpha Find the inverse function of $ f ^ {-1} $.

インボリュート逆関数.png

Please check the introductory section for details. Various methods to numerically create the inverse function of a certain function Introduction --Qiita

How to represent the inverse function

The first method I came up with was a polynomial approximation. The inverse function I want to find is a monotonically increasing function, and I thought that even a polynomial could be approximated with reasonable accuracy. That is, the inverse function $ y = a_0 + a_1x + a_2x^{2} + ...$ By expressing it like this, it is an idea that a close value can be obtained.

How to find a polynomial

The problem is how to determine the coefficients $ a_0, a_1, a_2, ... $ of the polynomial. You need to determine the coefficients so that the difference between the $ y $ value obtained by the polynomial and the true value is minimized. scikit-learn is a library for machine learning, but I found that it can be used to automatically determine the coefficients of polynomials.

Generation of training data

Use the involute function to generate training data for training.

Notebook


def involute(α):
    return np.tan(α) - α

Notebook


y = np.linspace(- np.pi / 4, np.pi / 4, 1000)
x = involute(y)

According to machine learning conventions, the input ($ inv \ alpha $) is $ x $ and the output ($ \ alpha $) is $ y $. Prepare the value of $ y $ ($\ alpha $) first, find the value of $ x $ ($ inv \ alpha $) using the involute function, and use it as training data.

Fitting polynomials with scikit-learn

You can easily fit polynomials with scikit-learn.

First, import the required libraries.

Notebook


from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

The training data generated above is a one-dimensional array, but in scikit-learn, the data is basically a column vector, so it is converted to a column vector.

Notebook


x_column = x.reshape(-1, 1)
y_column = y.reshape(-1, 1)

First, let's fit a 10th order polynomial.

Notebook


model_poly = make_pipeline(PolynomialFeatures(degree=10), LinearRegression())
model_poly.fit(x_column, y_column)
model_poly.score(x_column, y_column)

output


0.95581676585298314

PolynomialFeatures (degree = 10) is the conversion to a 10th-order polynomial, and LinearRegression () is the generation of a linear regression model, which are combined with the make_pipeline function to create a polynomial regression model. The training data is given to the fit method of the created model to fit the polynomial. The score method numerically evaluates the degree of estimation accuracy. If this value is 1.0, it means that it can be estimated perfectly.

Plot of estimates by polynomial regression model

Now let's plot the values estimated by the polynomial regression model.

Notebook


y_pred = model_poly.predict(x_column).flatten()
fig = figure(width=400, height=400)
fig.scatter(x, np.degrees(y), size=1, legend='true value')
fig.line(x, np.degrees(y_pred), line_color='orange', legend='Estimated value')
fig.xaxis.axis_label = 'invα'
fig.yaxis.axis_label = 'Pressure angle α(deg)'
fig.legend.location = 'top_left'
show(fig)
インボリュート逆関数_10次多項式.png

The input value is given to the predict method to obtain the estimated value. The graph above is the result of plotting the obtained estimates. The estimation accuracy is too bad.

How about increasing the order to 20?

Notebook


model_poly = make_pipeline(PolynomialFeatures(degree=20), LinearRegression())
model_poly.fit(x_column, y_column)
model_poly.score(x_column, y_column)

output


0.97492606041826035

Notebook


y_pred = model_poly.predict(x_column).flatten()
fig = figure(width=400, height=400)
fig.scatter(x, np.degrees(y), size=1, legend='true value')
fig.line(x, np.degrees(y_pred), line_color='orange', legend='Estimated value')
fig.xaxis.axis_label = 'invα'
fig.yaxis.axis_label = 'Pressure angle α(deg)'
fig.legend.location = 'top_left'
show(fig)
インボリュート逆関数_20次多項式.png

Even if it is increased to the 20th order, the wave of fluctuation is only small, and there is not much improvement. It is unlikely that it will be better to increase the order as it is.

Why polynomial regression cannot estimate well

I think that the large gradient near the origin is the reason why it cannot be estimated well by polynomial regression. The involute inverse function has an infinite gradient at the origin, but as long as you use a polynomial, you can never express an infinite gradient.

Source code

The Notebook used for the explanation is uploaded to Gist. Involute Inverse Function Estimation_Polynomial Regression.ipynb

Recommended Posts

Various methods to numerically create the inverse function of a certain function Part 1 Polynomial regression
Various methods to numerically create the inverse function of a certain function Introduction
Create a function to get the contents of the database in Go
How to create a wrapper that preserves the signature of the function to wrap
Create a function to visualize / evaluate the clustering result
I made a function to check the model of DCGAN
Find the optimal value of a function with a genetic algorithm (Part 2)
[Go] Create a CLI command to change the extension of the image
[Python3] Define a decorator to measure the execution time of a function
Finding the optimum value of a function using a genetic algorithm (Part 1)
[Python] A simple function to find the center coordinates of a circle
4 methods to count the number of occurrences of integers in a certain interval (including imos method) [Python implementation]
Python: I want to measure the processing time of a function neatly
I made a function to see the movement of a two-dimensional array (Python)
10 methods to improve the accuracy of BERT
Various ways to create a dictionary (memories)
Various ways to read the last line of a csv file in Python
[Circuit x Python] How to find the transfer function of a circuit using Lcapy
Use twitter API to get the number of tweets related to a certain keyword
[Verification] Try to align the point cloud with the optimization function of pytorch Part 1
I tried to create a model with the sample of Amazon SageMaker Autopilot
How to calculate the volatility of a brand
[Python] List Comprehension Various ways to create a list
How to create a function object from a string
[python] Create a list of various character types
Get the caller of a function in Python
Create a command to get the work log
Try to create a battle record table with matplotlib from the data of "Schedule-kun"
[Ruby] How to replace only a part of the string matched by the regular expression?
Notification of weather forecast (rain, etc.) by DM as a part of the function of bot
Add a function to tell the weather of today to slack bot (made by python)
Read the Python-Markdown source: How to create a parser
Create a dataset of images to use for learning
Explanation of the concept of regression analysis using python Part 2
A memo to visually understand the axis of pandas.Panel
Cut a part of the string using a Python slice
How to create a submenu with the [Blender] plugin
A python implementation of the Bayesian linear regression class
# Function that returns the character code of a string
Explanation of the concept of regression analysis using Python Part 1
Steps to calculate the likelihood of a normal distribution
Tweet the probability of precipitation as part of the function of the bot
I tried to erase the negative part of Meros
How to hit the document of Magic Function (Line Magic)
Create a shape on the trajectory of an object
Python Note: The mystery of assigning a variable to a variable
[Linux] [C / C ++] How to get the return address value of a function and the function name of the caller
A super introduction to Django by Python beginners! Part 6 I tried to implement the login function
What seems to be a template of the standard input part of the competition pro in python3
I tried to create a Python script to get the value of a cell in Microsoft Excel
I made a function to crop the image of python openCV, so please use it.
When a character string of a certain series is in the Key of the dictionary, the character string is converted to the Value of the dictionary.
Create a 2D array by adding a row to the end of an empty array with numpy