Understand machine learning ~ ridge regression ~.
Introduction
Nice to meet you. It is m0rio0818. I recently started studying machine learning. I'm still groping, but I'd like to keep it here as a memorandum.
I'm not good at SNS in general, so I didn't continue for the first time, but I'll do my best to continue posting. I am studying on my own and I think there are some mistakes. I would be grateful if you could point out.
Least squares
First, I would like to briefly touch on the least squares method. In machine learning regression analysis, formulas are created using the theory of least squares. As a least squares image, take a plot and draw a straight line that runs through the middle of each data. Let the linear expression be y = ax + b. For the time being, I tried to draw a straight line to make an image. The image is shown below.
There is a discrepancy between this prediction and the actual situation. The error between this straight line and each point is e1, e2, e3, e4, e5 in the above figure. Square the error and divide by 5.
E = \frac{e1^2 + e2^2 + e3^2 + e4^2 + e5^2}{5}
The smaller this error E is, the better the regression line. In addition, the least squares method does not impose restrictions on a and b when creating a prediction formula (regression formula).
Ridge regression
Next is about ridge regression. Ridge regression is a slightly modified version of the least squares method mentioned earlier. In particular, -The least squares method deals only with error E.
-In ridge regression, consider F, which is the sum of the error E and the square of the coefficient. F is also called a "regularization term".
F = {a^2 + b^2}
Then, with L = E + F, calculate the coefficients a and b that minimize L. In reality, the degree of influence of F can be adjusted by multiplying the regularization term by a constant, such as L = E + 0.2 × F.
Ridge regression is one of the regularized linear regressions, which is "a linear regression plus the square of the learned weight".
In addition, ridge regression can prevent overfitting.
Finally
I used the following references.
I got a little understanding of what ridge regression looks like. Next time, I would like to implement ridge regression.
References
Books Machine learning with python that you can understand clearly
website https://aizine.ai/ridge-lasso-elasticnet/