Understand machine learning ~ ridge regression ~.

Introduction

Nice to meet you. It is m0rio0818. I recently started studying machine learning. I'm still groping, but I'd like to keep it here as a memorandum.

I'm not good at SNS in general, so I didn't continue for the first time, but I'll do my best to continue posting. I am studying on my own and I think there are some mistakes. I would be grateful if you could point out.

Least squares

First, I would like to briefly touch on the least squares method. In machine learning regression analysis, formulas are created using the theory of least squares. As a least squares image, take a plot and draw a straight line that runs through the middle of each data. Let the linear expression be y = ax + b. For the time being, I tried to draw a straight line to make an image. The image is shown below.

図1.png

There is a discrepancy between this prediction and the actual situation. The error between this straight line and each point is e1, e2, e3, e4, e5 in the above figure. Square the error and divide by 5.

E = \frac{e1^2 + e2^2 + e3^2 + e4^2 + e5^2}{5}

The smaller this error E is, the better the regression line. In addition, the least squares method does not impose restrictions on a and b when creating a prediction formula (regression formula).

Ridge regression

Next is about ridge regression. Ridge regression is a slightly modified version of the least squares method mentioned earlier. In particular, -The least squares method deals only with error E.

-In ridge regression, consider F, which is the sum of the error E and the square of the coefficient. F is also called a "regularization term".

 F = {a^2 + b^2}

Then, with L = E + F, calculate the coefficients a and b that minimize L. In reality, the degree of influence of F can be adjusted by multiplying the regularization term by a constant, such as L = E + 0.2 × F.

Ridge regression is one of the regularized linear regressions, which is "a linear regression plus the square of the learned weight".

In addition, ridge regression can prevent overfitting.

Finally

I used the following references.

I got a little understanding of what ridge regression looks like. Next time, I would like to implement ridge regression.

References

Books Machine learning with python that you can understand clearly

website https://aizine.ai/ridge-lasso-elasticnet/

Recommended Posts

Understand machine learning ~ ridge regression ~.
Machine learning logistic regression
Machine learning linear regression
Machine Learning: Supervised --Linear Regression
Supervised machine learning (classification / regression)
Machine learning stacking template (regression)
Machine learning algorithm (logistic regression)
Machine learning
Machine learning beginners try linear regression
Machine learning algorithm (simple regression analysis)
Classification and regression in machine learning
<Course> Machine Learning Chapter 3: Logistic Regression Model
Machine learning algorithm (generalization of linear regression)
Record the steps to understand machine learning
Machine learning with python (2) Simple regression analysis
<Course> Machine Learning Chapter 1: Linear Regression Model
[Memo] Machine learning
<Course> Machine Learning Chapter 2: Nonlinear Regression Model
Machine learning classification
Stock price forecast using machine learning (regression)
Understand and implement ridge regression (L2 regularization)
Machine learning algorithm (linear regression summary & regularization)
[Machine learning] Regression analysis using scikit learn
Machine Learning sample
(Machine learning) I tried to understand Bayesian linear regression carefully with implementation.
EV3 x Python Machine Learning Part 2 Linear Regression
[Python3] Let's analyze data using machine learning! (Regression)
Machine learning tutorial summary
About machine learning overfitting
Machine learning ⑤ AdaBoost Summary
Machine Learning: Supervised --AdaBoost
Machine learning support vector machine
Studying Machine Learning ~ matplotlib ~
Machine learning course memo
Machine learning library dlib
Machine learning (TensorFlow) + Lotto 6
Somehow learn machine learning
Supervised learning (regression) 1 Basics
Python: Supervised Learning (Regression)
Machine learning library Shogun
Machine learning rabbit challenge
Introduction to machine learning
Machine Learning: k-Nearest Neighbors
What is machine learning?
Coursera Machine Learning Challenges in Python: ex2 (Logistic Regression)
Python Scikit-learn Linear Regression Analysis Nonlinear Simple Regression Analysis Machine Learning
Coursera Machine Learning Challenges in Python: ex1 (Linear Regression)
Artificial intelligence, machine learning, deep learning to implement and understand
Machine learning model considering maintainability
Machine learning learned with Pokemon
Data set for machine learning
Japanese preprocessing for machine learning
Try to evaluate the performance of machine learning / regression model
An introduction to machine learning
Machine learning / classification related techniques
Basics of Machine Learning (Notes)
Ridge regression with Pyspark's Mllib
[Machine learning] Understanding random forest
Machine learning with Python! Preparation
Machine Learning Study Resource Notepad
Machine learning ② Naive Bayes Summary