Numerical approximation method when the calculation of the derivative is troublesome

It is a method to approximate by numerical calculation when it is troublesome to calculate or implement the derivative of the formula you want to find by the steepest descent method or the Levenberg-Marquardt method. It seems that the following three types are often used, and it seems that you should decide by considering the balance between the amount of calculation and the accuracy.

Approximation method

Two-point approximation

The simplest approximation and the fastest calculation. The accuracy is reasonable. Expressed in a mathematical formula

f'(x_0) \approx \frac{f(x_0+h)-f(x_0)}{h}

However, $ h $ is small enough. When implemented in C language,

#include <float.h>

extern double f( double x );

double df( double x )
{
    const double h = FLT_EPSILON;
    double y1 = f( x + h );
    double y2 = f( x );

    return ( y1 - y2 ) / h;
}

It seems that $ h $ can be calculated positively or negatively, but the result will change.

About FLT_EPSILON

FLT_EPSILON is the value defined in float.h. It is a small value and is defined as the value of the difference between 1 and the "minimum number greater than 1" called mechanical epsilon. FLT_EPSILON itself is a mechanical epsilon for float, but if the value of x is small, rounding error is likely to occur, so I am using this for the time being, not for double.

There is also a definition for double, and DBL_EPSILON is defined in the same float.h.

3-point approximation (center difference)

If it is a two-point approximation, even the point $ x_0 $ where the derivative value becomes zero is unlikely to become zero, so it is a method to improve it a little and use the points before and after the point $ x_0 $. The amount of calculation is slightly higher than the two-point approximation, and the accuracy is improved.

Expressed in a mathematical formula

f'(x_0) \approx \frac{f(x_0+h)-f(x_0-h)}{2h}

When implemented in C language,

#include <float.h>

extern double f( double x );

double df( double x )
{
    const double h = FLT_EPSILON;
    double y1 = f( x + h );
    double y2 = f( x - h );

    return ( y1 - y2 ) / ( 2 * h );
}

Although it is called a 3-point approximation, only 2 points appear in the calculation. Considering the mean value theorem, it seems more likely that this is closer to the true value of x.

The result is the same regardless of whether $ h $ is positive or negative.

5-point approximation

f'(x_0) \approx \frac{f(x_0-2h)-8f(x_0-h)+8f(x_0+h)-f(x_0+2h)}{12h}

It became an expression that I could not understand at once. It seems to be derived by making full use of Taylor expansion and Lagrange interpolation polynomial.

When implemented in C language,

#include <float.h>

extern double f( double x );

double df( double x )
{
    const double h = FLT_EPSILON;
    double y1 = f( x + h );
    double y2 = f( x - h );
    double y3 = f( x + 2 * h );
    double y4 = f( x - 2 * h );

    return ( y4 - 8 * y2 + 8 * y1 - y3 ) / ( 12 * h );
}

I don't know what it is anymore, so I'm grateful to use it, just remembering that it can be calculated this way.

Partial differential

If it is a function made up of multiple variables, it will be partially differentiated, but the same thing is done. If you want to partially differentiate $ y $ of $ f (x, y, z) $ with a two-point approximation

f_y(x_0,y_0,z_0) \approx \frac{f(x_0,y_0+h,z_0)-f(x_0,y_0,z_0)}{h}

You will implement this.

Recommended Posts

Numerical approximation method when the calculation of the derivative is troublesome
About the accuracy of Archimedean circle calculation method
[Scientific / technical calculation by Python] Numerical calculation to find the value of derivative (differential)
Numerical calculation of compressible fluid by finite volume method
Calculation of the shortest path using the Monte Carlo method
What is the true identity of Python's sort method "sort"? ??
When you think the update of ManjaroLinux is strange
The copy method of pandas.DataFrame is deep copy by default
Unfortunately there is no sense of unity in the where method
Play with numerical calculation of magnetohydrodynamics
Is the probability of precipitation correct?
Science "Is Saito the representative of Saito?"
I replaced the numerical calculation of Python with Rust and compared the speed
Find out the name of the method that called it from the method that is python
[Introduction to Python] What is the method of repeating with the continue statement?
A method to check the maximum number of digits after the decimal point when entering pd.Series containing a numerical value
Count / verify the number of method calls.
What is the cause of the following error?
Numerical calculation of differential equations with TensorFlow 2.0
When the target is Ubuntu 16.04 in Ansible
The update of conda is not finished.
The backslash of the Japanese keyboard is "ro"
[pandas] When specifying the default Index label in the at method, "" is not required
It seems that the version of pyflakes is not the latest when flake8 is installed
In Python, change the behavior of the method depending on how it is called
Approximation by the least squares method of a circle with two fixed points
Differences in the behavior of each LL language when the list index is skipped