Machine learning ⑤ AdaBoost Summary

Summary of AdaBoost

What is AdaBoost?

AdaBoost is a machine learning model that attempts to create a strong discriminator by combining weak discriminators that are slightly more accurate than random. The flow of how to make First, apply a weak discriminator and increase the weight of those that have been misclassified. Then, the ones with the weight are given priority and classified. I repeat that.

It is easy to understand if you refer to the figure below. I also posted a Youtube link, so if you want to know more, please take a look.

Screen Shot 2017-05-13 at 12.44.42.png Extracted from Alexander Ihler's youtube video

In the figure above, D1 first uses a weak classifier to classify and then D2 misclassifies'+' 1 and'-' 2 weights. Next, the three misclassified items are prioritized and classified again. Here, at the same time as increasing the weight, the weight of others that are correctly classified is decreasing. Furthermore, in D3, the weights of the'-'3 misclassified in D2 are increased and at the same time the weights of others are decreased. By the way, in AdaBoost's default code, the Decision Tree is used for classification by this weak discriminator.

Based on the weight of the repeated classification, we will make a strong discriminator.

Screen Shot 2017-05-13 at 13.00.11.png Extracted from Alexander Ihler's youtube video

default code

python



from sklearn.ensemble import AdaBoostClassifier

AdaBoostClassifier(base_estimator=None, n_estimators=50, learning_rate=1.0, algorithm='SAMME.R', random_state=None)

A description of the Parameters in AdaBoost

A machine learning model used as a weak discriminator. DecisionTreeClassifier is used for default, but other machine learning models can be adapted by specifying it.

Specify how many times to repeat the classification using a weak discriminator. However, before that, if the strong discriminator becomes 100% accurate, it ends there.

The pros and cons of AdaBoost.

Since some weak classifiers are combined, it is easy to classify accurately.

--Bad point

Same as K nearest neighbors, it is also vulnerable to noise (different labels are mixed in the same place) and Outliers (outliers). Prone to overfitting.

Summary

The above is an overview of AdaBoost as far as I can tell. We will update it daily, so if you have something to add or fix, we would appreciate it if you could comment.

Recommended Posts

Machine learning ⑤ AdaBoost Summary
Machine learning tutorial summary
Machine Learning: Supervised --AdaBoost
Machine learning ② Naive Bayes Summary
Machine learning
Machine learning article summary (self-authored)
Machine learning ④ K-nearest neighbor Summary
Machine learning ① SVM (Support Vector Machine) Summary
Machine learning summary by Python beginners
Machine learning ③ Summary of decision tree
[Memo] Machine learning
Machine learning classification
Machine Learning sample
scikit-learn How to use summary (machine learning)
"Python Machine Learning Programming" Summary Note (Jupyter)
Machine learning algorithm classification and implementation summary
Machine learning algorithm (linear regression summary & regularization)
About machine learning overfitting
Machine learning logistic regression
Machine learning support vector machine
Studying Machine Learning ~ matplotlib ~
Machine learning linear regression
Machine learning course memo
Machine learning library dlib
Machine learning (TensorFlow) + Lotto 6
Summary of evaluation functions used in machine learning
Somehow learn machine learning
Summary for learning RAPIDS
Machine learning library Shogun
Machine learning rabbit challenge
Introduction to machine learning
Machine Learning: k-Nearest Neighbors
What is machine learning?
Machine Learning Professional Series Round Reading Session Slide Summary
Machine learning python code summary (updated from time to time)
Site summary to learn machine learning with English video
Summary of the basic flow of machine learning with Python
Machine learning model considering maintainability
Machine learning learned with Pokemon
Data set for machine learning
Machine learning in Delemas (practice)
An introduction to machine learning
Machine learning / classification related techniques
Machine Learning: Supervised --Linear Regression
Ensemble learning summary! !! (With implementation)
Basics of Machine Learning (Notes)
Machine learning beginners tried RBM
[Machine learning] Understanding random forest
Machine learning with Python! Preparation
Machine Learning Study Resource Notepad
Understand machine learning ~ ridge regression ~.
About machine learning mixed matrices
Machine Learning: Supervised --Random Forest
Practical machine learning system memo
Machine learning Minesweeper with PyTorch
Machine learning environment construction macbook 2021
Build a machine learning environment
Python Machine Learning Programming> Keywords
Machine learning algorithm (simple perceptron)
Used in machine learning EDA
Importance of machine learning datasets