Reference: [Deep Learning from scratch (p98 4.3.1)](https://www.amazon.co.jp/%E3%82%BC%E3%83%AD%E3%81%8B%E3%82 % 89% E4% BD% 9C% E3% 82% 8BDeep-Learning-% E2% 80% 95Python% E3% 81% A7% E5% AD% A6% E3% 81% B6% E3% 83% 87% E3% 82% A3% E3% 83% BC% E3% 83% 97% E3% 83% A9% E3% 83% BC% E3% 83% 8B% E3% 83% B3% E3% 82% B0% E3% 81% AE% E7% 90% 86% E8% AB% 96% E3% 81% A8% E5% AE% 9F% E8% A3% 85-% E6% 96% 8E% E8% 97% A4-% E5% BA% B7% E6% AF% 85 / dp / 4873117585)
What should I do when I want to differentiate the function f at the x point? If you try to calculate like a mathematical formula, it will be as follows.
＞ Ideal if this calculation can be done
> But due to rounding error np.fload32(10e-50)=0.It becomes 0.
def numerical_diff(f, x):
h = 10e-50
return ( f(x+h) - f(x) ) / h
So, do as follows.
def numerical_diff(f, x):
h = 1e-4 # 0.00001
return ( f(x+h) - f(x) ) / h
However, in this case, the difference between x points and (x + h / 2) points is 0.00001. It becomes the derivative of the point (x + 0.000005) between them. Since the difference between x and x + h has not been minimized, there is a difference. It cannot be said to be the derivative at point x. The following is the derivative of the function f at the x point.
def numerical_diff(f, x):
h = 1e-4 # 0.00001
return ( f(x+h) - f(x-h) ) / (2*h)
By the way, finding the derivative by the difference when a minute value is given is called numerical differentiation.
import numpy as np
import matplotlib.pylab as plt
def function(x):
return 0.01*x**2 + 0.1*x
def nd1(f, x):
h = 1e-4 # 0.00001
return ( f(x+h) - f(x) ) / h
def nd2(f, x):
h = 1e-4 # 0.00001
return ( f(x+h) - f(x-h) ) / (2*h)
#0 in true derivative.2,0.It will be 3, so the closer it is, the better.
#For nd1
print(nd1(function, 5)) # => 0.20000099999917254
print(nd1(function, 10)) # => 0.3000009999976072
#For nd2
print(nd2(function, 5)) # => 0.1999999999990898
print(nd2(function, 10)) # => 0.2999999999986347