Generalized linear model (GLM) and neural network are the same (1)

It sounds a bit rough, but it's the same thing as a generalized linear model (GLM) and a neural network multi-layer perceptron. In the scattered data, I'm going to draw a convincing line. I think there are many people who do both fields, but I think it's easier to understand at once.

In each of the statistical modeling and discriminant functions, ・ How about drawing a straight line in any data with a linear model (LM)? → Let's generalize (GLM) so that non-linear lines can be drawn.

From the perspective of statistical modeling: Generalized Linear Models (GLM)

For Linear Model and Generalized Linear Model, The materials for the statistical solution study session are summarized here, which is quite easy to understand. " Statistical Study Group Materials-Day 2 Generalized Linear Models-: Logics of Blue "

So, if you write a linear model, a simple regression model or a multiple regression model in general,


{ \displaystyle
y_i = \alpha + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 … + \beta_i x_i + \varepsilon_i
}

When written in a matrix,


\begin{equation}

\left(
    \begin{array}{c}
      y_1 \\
      y_2 \\
・\\
・\\
      y_n
    \end{array} 

\right)

=

\left(

 \begin{array}{ccc}
      1 & x_{11} & x_{12} \\
      1 & x_{12} & x_{22} \\
・&・&・\\
・&・&・\\
      1 & x_{n1} & x_{n2}
    \end{array}
\right)

\left(
   \begin{array}{c}
     \alpha \\
     \beta_1 \\
・\\
・\\
     \beta_n 
    \end{array}
\right)

+

\left(
   \begin{array}{c}
     \varepsilon_1 \\
     \varepsilon_2 \\
・\\
・\\
     \varepsilon_n 
    \end{array}
\right)


\end{equation}

further

y = B X + \epsilon

Can also be written. This time, for the sake of simplicity, the explanation variate is set to 2.

y = \beta_1 x_1 + \beta_2 x_2 = BX

Think about.

Next, in order to remove the constraint of assuming only a straight line and a normal distribution from the above equation, Use the link function on the left side of the regression equation. Assuming a Bernoulli distribution, using the logit link function

\log \frac{y}{1-y} = \beta_1 x_1 + \beta_2 x_2

The left side is the link function and the right side is the linear predictor. When this formula is converted by the inverse function,

  y = \frac{1}{1+exp(-(\beta_1 x_1 + \beta_2 x_2))} = \frac{1}{1+exp(-BX)}  (1)

Can be obtained.

In statistical modeling, after this, parameters are used using a likelihood function, etc.

\beta_1  \beta_2
```To build a model by estimating=Draw a line in the data.

 It's getting longer, so I'll continue in the next article.


Recommended Posts

Generalized linear model (GLM) and neural network are the same (1)
Generalized linear model (GLM) and neural network are the same (2)
Python open and io.open are the same
[Python3] "A // B" and "math.floor (A / B)" are not always the same! ??
[PyTorch] Make sure the model and dataset are in cuda mode
I tried using GLM (generalized linear model) for stock price data
Estimator calculation / prediction at Lasso and Ridge of generalized linear model
Neural network with OpenCV 3 and Python 3
Simple classification model with neural network
Simple neural network theory and implementation
Touch the object of the neural network
[For beginners] Why are the "weights" and "bias" of neural networks necessary?
Persist the neural network built with PyBrain
2. Mean and standard deviation with neural network!