Significance of machine learning and mini-batch learning

Introduction

This is a note about mini-batch learning in machine learning. It also serves as an intuitive explanation of machine learning.

What is machine learning?

Machine learning is an arithmetic unit that automatically extracts rules that are assumed to be inherent in a given data. That is, when the output $ \ boldsymbol {t} $ is already obtained for a certain input set $ \ boldsymbol {x} $, a new input set $ \ is extracted by extracting the rules that exist there. Expect $ \ boldsymbol {t} ^ {\ prime} $ corresponding to boldsymbol {x} ^ {\ prime} . this**Rules**What will be(Called weight)Generally a matrixWThen, the above story is the output of machine learning\boldsymbol{y}(Ideally\boldsymbol{t}Should match)To$ \boldsymbol{y} = W \boldsymbol{x}\tag{1}$When\boldsymbol{y}When\boldsymbol{t}Loss function obtained from$ L = \frac{1}{2}||\boldsymbol{y}(\boldsymbol{x},W)-\boldsymbol{t}||^{2}\tag{2}$To可能な限り小さくするようなW$To求めるWhenいうこWhenに置き換えられます。

Big data and mini-batch learning

Now, suppose that big data $ (\ boldsymbol {x} \ _ {n}, \ boldsymbol {t} \ _ {n}) $ is obtained again ($ n = 1,2, \ cdots, N $). $ N $ is big enough). Extract the weight $ W $ from this and predict the appropriate output $ \ boldsymbol {t} ^ {\ prime} $ for the input $ \ boldsymbol {x} \ ^ {\ prime} $ whose output is unknown. Therefore, the sum of squares error for all $ \ boldsymbol {x} \ _ {n} $ $\frac{1}{N} \sum_{n=1}^{N}\|\|\boldsymbol{y}\_{n}(\boldsymbol{x}\_{n},W)-\boldsymbol{t}_{n} \|\|^{2} \tag{3}Like to minimizeW$Ask for. For example, there are methods such as gradient descent.

The scale of big data we are dealing with nowN(\gg 1)So even if it's a computerWTo seekn=1FromNIt is not a wise method because the amount of calculation is enormous if the differential calculation is done honestly. ThereforeNOfM (\ll N)Only one piece of data is randomly taken out and against this$ \frac{1}{M} \sum_{m=1}^{M}\|\|\boldsymbol{y}\_{m}(\boldsymbol{x}\_{m},W)-\boldsymbol{t}_{m} \|\|^{2} \tag{4}By minimizing(3)Overwhelmingly efficient with respect to the method ofWCan be determined. Obtained in this wayW$は、与えられた全てのデータFrom求まったわけではありませんが、もとのデータのもつ規則の良い近似となっていることが多いです。このような学習手法をミニバッチ学習と呼びます。

Recommended Posts

Significance of machine learning and mini-batch learning
Numerai Tournament-Fusion of Traditional Quants and Machine Learning-
Basics of Machine Learning (Notes)
Importance of machine learning datasets
Machine learning and mathematical optimization
Machine learning ③ Summary of decision tree
Classification and regression in machine learning
Machine learning
Organize machine learning and deep learning platforms
Summary of recommended APIs for artificial intelligence, machine learning, and AI
[Machine learning] OOB (Out-Of-Bag) and its ratio
Machine learning algorithm (generalization of linear regression)
Meaning of deep learning models and parameters
2020 Recommended 20 selections of introductory machine learning books
Machine learning algorithm (implementation of multi-class classification)
Personal notes and links about machine learning ① (Machine learning)
Machine learning algorithm classification and implementation summary
Python and machine learning environment construction (macOS)
[Machine learning] List of frequently used packages
"OpenCV-Python Tutorials" and "Practical Machine Learning System"
Judgment of igneous rock by machine learning ②
[Memo] Machine learning
Machine learning classification
Basic machine learning procedure: ③ Compare and examine the selection method of features
Python: Preprocessing in machine learning: Handling of missing, outlier, and imbalanced data
Machine Learning sample
Summary of mathematical scope and learning resources required for machine learning and data science
Machine learning memo of a fledgling engineer Part 1
Classification of guitar images by machine learning Part 1
Study machine learning and computer science. Resource list
Beginning of machine learning (recommended teaching materials / information)
Machine learning of sports-Analysis of J-League as an example-②
Python & Machine Learning Study Memo ⑤: Classification of irises
A memorandum of studying and implementing deep learning
Python & Machine Learning Study Memo ②: Introduction of Library
Full disclosure of methods used in machine learning
Machine learning Training data division and learning / prediction / verification
List of links that machine learning beginners are learning
Parallel learning of deep learning by Keras and Kubernetes
Overview of machine learning techniques learned from scikit-learn
About the development contents of machine learning (Example)
Summary of evaluation functions used in machine learning
Analysis of shared space usage by machine learning
[Translation] scikit-learn 0.18 Tutorial Introduction of machine learning by scikit-learn
Machine learning memo of a fledgling engineer Part 2
Reasonable price estimation of Mercari by machine learning
Classification of guitar images by machine learning Part 2
Get a glimpse of machine learning in Python
Try using Jupyter Notebook of Azure Machine Learning
Arrangement of self-mentioned things related to machine learning
Causal reasoning using machine learning (organization of causal reasoning methods)
Evaluation method of machine learning regression problem (mean square error and coefficient of determination)
[For beginners of artificial intelligence] Machine learning / Deep Learning Programming Learning path and reference books
Machine Learning: Image Recognition of MNIST by using PCA and Gaussian Native Bayes
Predict short-lived works of Weekly Shonen Jump by machine learning (Part 2: Learning and evaluation)
List of main probability distributions used in machine learning and statistics and code in python
Machine learning tutorial summary
Machine learning with Raspberry Pi 4 and Coral USB Accelerator
Key points of "Machine learning with Azure ML Studio"
Mayungo's Python Learning Note: List of stories and links
About machine learning overfitting