Deep learning 1 Practice of deep learning

Aidemy 2020/9/30

Introduction

Hello, it is Yope! I am a liberal arts student, but I was interested in the possibilities of AI, so I went to the AI-specialized school "Aidemy" to study. I would like to share the knowledge gained here with you, and I am summarizing it on Qiita. I am very happy that many people have read the previous summary article. Thank you! This is the first post of deep learning. Nice to meet you.

What to learn this time ・ About deep learning ・ Create a model to classify handwritten numbers

About deep learning

What is deep learning?

-Deep learning is a type of machine learning that uses a model called deep neural network to classify and regress data. It is called by such a name because it is made with reference to the nerve (neuron) network of living organisms. ・ Deep learning is widely used in the fields of image recognition and voice recognition. The purpose of deep learning is to "improve the recognition accuracy of images and sounds."

-The (deep) neural network is composed of layers of elements that receive and output information called neurons. A model with three or more layers at this time is treated as a model called a deep neural network. At this time, the input layer is called input layer, the output layer is called output layer, and the layer between the two is called hidden layer. The layer in which the previous neuron and the next neuron are connected for each layer is called __fully connected layer __.

-The neuron receives the input values called __input value (x) __ and __ weight parameter (w) __. If the value of the expression represented by x and w is higher than the __threshold (θ) __ of each neuron, the value of 1 is passed to the next neuron, otherwise 0 is passed. This information transmission is called __neuron firing __. ・ In deep learning, a classification model or regression model is created by mechanically adjusting the value of the weight parameter (w).

Flow of deep learning

・ Create a network model → Give the model training data to train → (learning completed) → implement

・ First, as confirmed in the previous section, a network model is constructed by forming a hierarchical structure of some neurons. Next, training data (input value x) is given to the created model and the output value y is output. At first, the value of this y is different from the examination, but the value of the weight parameter (w) is adjusted so that the deviation ΔE from the correct answer data (teacher label) becomes small, and finally the appropriate value is set. The model to be returned is completed.

Create a model to classify handwritten numbers

Flow of classification (model implementation)

・ Data preparation → Neural network model construction → Data is given and trained → Model accuracy evaluation

Supplement about deep neural networks

-This time, when the input handwritten data is "7", the value corresponding to "7" among the correct answer labels of the teacher data is 1 (on), and the other values are 0 (off). Data with one correct answer and 0 output other than this is called __one-hot vector __. Also, the assignment (label) when classifying data such as "7" is called __class label __.

-In the case of a handwritten character image, the input data is composed of a set of 28 * 28 numbers (vector). The element that receives and calculates this vector is called __node (unit) __, and the vertical unit group is called __layer (layer) __.

TensorFlow and Keras

-TensorFlow is a library for machine learning provided by Google. ・ Keras can make TensorFlow code intuitive and concise. Wrapper libraries, like Keras, make it easier to use other libraries (TensorFlow in this case).

Data preparation

-Handwritten digit data can be used immediately by importing a Keras MNIST dataset. ・ (Review) X is image data and y is teacher label data. train is training data and test is test data for model evaluation.

from keras.datasets import mnist
(X_train,y_train),(X_test,y_test)=mnist.load_data()

Build a neural network model

-The method of building (creating) a model is to first create a __Sequential () __ instance that manages the model, and then create a __Sequential () __ instance. __add (Dense (number of output units in the first fully connected layer, input_dim = number of input units)) __ The input layer and the first fully connected layer are created by using. In addition, it is necessary to specify __activation function __ for the output of the fully connected layer. The method is __add (Activation ("function name")) __. If you want to create the second and subsequent input layers, you can specify the Dense value with the add () method each time (input_dim can be specified only once). Finally, set the learning process with the __compile () __ method and complete.

#Module import
from keras.model import Sequential
from keras.layers import Dense,Activation
#Creation of Sequential instance (model management)
model=Sequential()
#Number of input units 28*28=784, 256 output units of the first fully connected layer, activation function sigmoid
model.add(Dense(256,input_dim=784))
model.add(Activation("sigmoid"))
#256 output units of the second fully connected layer, activation function relu
model.add(Dense(128))
model.add(Activation("relu"))
#Number of output units of the third fully connected layer is 10, activation function softmax
model.add(Dense(10))
model.add(Activation("softmax"))
#Learning process settings (arguments are OK now I don't know)
model.compile(optimizer="sgd",loss="categorical_crossentropy",metrics=["accuracy"])

Model learning

-Learning with __model.fit (X_train, y_train, verbose = 0or1, epochs = number of learnings) __. ・ If epochs is 1, the progress of learning is output, and if it is 0, it is not output. ・ (Review) Learning is performed by changing the weight so that the difference between the output and the teacher data becomes small.

Model evaluation

-Evaluate whether the model can produce high accuracy other than training data by using test data. The accuracy of the test data at this time is called __generalization accuracy __. -Calculate the generalization accuracy with __model.evaluate (X_test, y_test, verbose = 0or1) __.

Use the trained (+ evaluated) model

・ Perform a classification problem using the learned (+ evaluated) model. ・ __Model.predict (self, input data, batch_size = integer, verbose = 0 or1, steps = number of steps) __ ・ About each argument

Regarding the return value, the predict () method returns a NumPy array that stores the predicted values, but since the handwritten character class is 10-dimensional, the largest value is extracted. The method uses __np.argmax (data using predict (), axis = 1) __ to get the index number of the maximum value. Since the index for each row can be obtained by setting axis = 1, only the prediction result is displayed.

Summary

-Deep learning refers to classifying and regressing data using a model called __deep neural network __. ・ In deep learning, a classification model or regression model is created by mechanically adjusting the value of the weight parameter (w). ・ The flow of deep learning is data preparation, model creation, model learning, model evaluation, and model classification. -Handwritten digit data can be used immediately by importing a Keras MNIST dataset. -Model creation, model learning, and model evaluation are performed with __add (), fit (), and evaluate () __, respectively. -Classification by model is done by __predict () __, but when actually outputting, use __np.argmax () __.

This time is over. Thank you for reading until the end.

Recommended Posts

Deep learning 1 Practice of deep learning
Deep running 2 Tuning of deep learning
Deep Learning
Deep reinforcement learning 2 Implementation of reinforcement learning
Deep Learning Memorandum
Start Deep learning
Python Deep Learning
Deep learning × Python
Othello-From the tic-tac-toe of "Implementation Deep Learning" (3)
Meaning of deep learning models and parameters
Try deep learning of genomics with Kipoi
Visualize the effects of deep learning / regularization
Sentiment analysis of tweets with deep learning
Learning record of reading "Deep Learning from scratch"
Othello-From the tic-tac-toe of "Implementation Deep Learning" (2)
First Deep Learning ~ Struggle ~
The story of doing deep learning with TPU
Python: Deep Learning Practices
Deep learning / activation functions
Deep Learning from scratch
An amateur tried Deep Learning using Caffe (Practice)
Deep learning / error back propagation of sigmoid function
A memorandum of studying and implementing deep learning
[Learning memo] Deep Learning from scratch ~ Implementation of Dropout ~
Deep learning / cross entropy
First Deep Learning ~ Preparation ~
Basic understanding of stereo depth estimation (Deep Learning)
First Deep Learning ~ Solution ~
[AI] Deep Metric Learning
I tried deep learning
Parallel learning of deep learning by Keras and Kubernetes
Python: Deep Learning Tuning
Deep learning large-scale technology
Implementation of Deep Learning model for image recognition
Deep learning learned by implementation (segmentation) ~ Implementation of SegNet ~
Deep learning / softmax function
Application of Deep Learning 2 made from scratch Spam filter
Techniques for understanding the basis of deep learning decisions
Othello ~ From the tic-tac-toe of "Implementation Deep Learning" (4) [End]
DNN (Deep Learning) Library: Comparison of chainer and TensorFlow (1)
Cats are already tired of loose fluffy deep learning
Collection and automation of erotic images using deep learning
DEEP PROBABILISTIC PROGRAMMING --- "Deep Learning + Bayes" Library --- Introduction of Edward
Deep Learning from scratch 1-3 chapters
Try deep learning with TensorFlow
Machine learning in Delemas (practice)
Deep Learning Gaiden ~ GPU Programming ~
<Course> Deep Learning: Day2 CNN
Basics of Machine Learning (Notes)
Reinforcement learning 2 Installation of chainerrl
Deep learning image recognition 1 theory
Other applications of dictionary learning
Deep learning / LSTM scratch code
Rabbit Challenge Deep Learning 1Day
<Course> Deep Learning: Day1 NN
Deep Kernel Learning with Pyro
Try Deep Learning with FPGA
Deep learning for compound formation?
Introducing Udacity Deep Learning Nanodegree
Supervised learning 1 Basics of supervised learning (classification)
Subjects> Deep Learning: Day3 RNN