Machine learning beginners tried RBM

at first...

This is a Qiita beginner's post. It's hard to see ... or anything like this, anyone can do it! Please forgive me ... To the last, it was done by Mr. Zero, who has no knowledge of machine learning and artificial intelligence. (At first, what was epoch? What was Accuracy? Ww)

What is RBM?

RBM(Restricted Boltzmann machine) It is a kind of pre-training method in deep learning, and it is a kind of model that makes a perfect match with AutoEncoder, which is often heard. The model was invented between 1984 and 1986, hoping for an edge in statistical mechanics. Unlike Autoencoder, which receives input and outputs deterministically, it is a generative model that can be discussed on a probability distribution, so it is known as a highly convenient model. スクリーンショット 2020-08-30 13.46.03.png

Deep Learning from RBM ~ With Black Magic ~

I'm not sure ...

I searched a lot!

Then ... There is something called BernoulliRBM in scikit learn! I have to do it

Implementation

For the time being, run it on a dataset called MNIST!

RBM.ipynb


import numpy as np
import matplotlib.pyplot as plt

from scipy.ndimage import convolve
from sklearn import linear_model, datasets, metrics
from sklearn.model_selection import train_test_split
from sklearn.neural_network import BernoulliRBM
from sklearn.pipeline import Pipeline
from sklearn.base import clone

#Data set used
from keras.datasets import fashion_mnist
from keras.datasets import mnist

from keras.layers import Input, Dense
from keras.models import Model
from keras import layers, models

import time
import numpy

For the time being, import looks like the above

Next, do as shown in the example (many parts do not know how to process)

RBM.ipynb


def nudge_dataset(X, Y):
    """
    This produces a dataset 5 times bigger than the original one,
    by moving the 8x8 images in X around by 1px to left, right, down, up
    """
    direction_vectors = [
        [[0, 1, 0],
         [0, 0, 0],
         [0, 0, 0]],

        [[0, 0, 0],
         [1, 0, 0],
         [0, 0, 0]],

        [[0, 0, 0],
         [0, 0, 1],
         [0, 0, 0]],

        [[0, 0, 0],
         [0, 0, 0],
         [0, 1, 0]]]

    def shift(x, w):
        return convolve(x.reshape((8, 8)), mode='constant', weights=w).ravel()

    X = np.concatenate([X] +
                       [np.apply_along_axis(shift, 1, X, vector)
                        for vector in direction_vectors])
    Y = np.concatenate([Y for _ in range(5)], axis=0)
    return X, Y


# Load Data
# (x_train, y_train), (x_test, y_test) = mnist.load_data()
# X, y = mnist.load_data()
# X = np.asarray(X, 'float32')
# X, Y = nudge_dataset(X, y)
# X = (X - np.min(X, 0)) / (np.max(X, 0) + 0.0001)  # 0-1 scaling

# X_train, X_test, Y_train, Y_test = train_test_split(
#     X, Y, test_size=0.2, random_state=0)

# (X_train, Y_train), (X_test, Y_test) = fashion_mnist.load_data()
(X_train, Y_train), (X_test, Y_test) = mnist.load_data()

X_train = X_train.astype('float32') / 255.
X_test = X_test.astype('float32') / 255.
X_train = X_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
X_test = X_test.reshape((len(x_test), np.prod(x_test.shape[1:])))

# Models we will use
logistic = linear_model.LogisticRegression(solver='newton-cg', tol=1)
rbm = BernoulliRBM(random_state=0, verbose=True)

rbm_features_classifier = Pipeline(
    steps=[('rbm', rbm), ('logistic', logistic)])

# #############################################################################
# Training

# Hyper-parameters. These were set by cross-validation,
# using a GridSearchCV. Here we are not performing cross-validation to
# save time.
rbm.learning_rate = 0.06
rbm.n_iter = 10
# More components tend to give better prediction performance, but larger
# fitting time
rbm.n_components = 100
logistic.C = 6000

# Training RBM-Logistic Pipeline
rbm_features_classifier.fit(X_train, Y_train)

# Training the Logistic regression classifier directly on the pixel
raw_pixel_classifier = clone(logistic)
raw_pixel_classifier.C = 100.
raw_pixel_classifier.fit(X_train, Y_train)

# #############################################################################
# Evaluation

Y_pred = rbm_features_classifier.predict(X_test)
print("Logistic regression using RBM features:\n%s\n" % (
    metrics.classification_report(Y_test, Y_pred)))

Y_pred = raw_pixel_classifier.predict(X_test)
print("Logistic regression using raw pixel features:\n%s\n" % (
    metrics.classification_report(Y_test, Y_pred)))

Sorry for the dirty source code ... I have to comment out ... It's difficult to write beautiful code ...

I am using Bernoulli RBM from the sklearn library. In this example, we are building a classification pipeline using a Bernoulli RBM feature extractor and a Logistic Regression classifier. For comparison, we present a logistic regression for raw pixel values.

[RBM source code (sklearn)](https://scikit-learn.org/stable/auto_examples/neural_networks/plot_rbm_logistic_classification.html#sphx-glr-auto-examples-neural-networks-plot-rbm-logistic-classification -py)

result

スクリーンショット 2020-08-30 14.03.44.pngスクリーンショット2020-08-3014.03.56.png

MNIST...Accuracy: 0.97 Fashion-MNIST...Accuracy: 0.79

If you increase the number of epochs, it will take a long time, so set the number of epochs to 10 (no time to submit the assignment ...)

Conclusion

I tried to copy and implement the source code, but it took a long time to find it ... I have to understand more! I thought. After that, a graph? I thought it would be easier to understand if I could display the loss function and so on.

I will post it little by little from now on. I want to post docker and web related ... I look forward to working with you.

Recommended Posts

Machine learning beginners tried RBM
Machine learning
I tried machine learning with liblinear
Machine learning summary by Python beginners
Machine learning beginners try linear regression
<For beginners> python library <For machine learning>
Machine learning beginners take Coursera's Deep learning course
First Steps for Machine Learning (AI) Beginners
[Memo] Machine learning
Machine learning classification
Machine Learning sample
Recommended study order for machine learning / deep learning beginners
List of links that machine learning beginners are learning
[For beginners] Introduction to vectorization in machine learning
Machine learning tutorial summary
About machine learning overfitting
Machine learning ⑤ AdaBoost Summary
Machine Learning: Supervised --AdaBoost
Machine learning logistic regression
Machine learning support vector machine
Machine learning beginners tried to make a horse racing prediction model with python
Studying Machine Learning ~ matplotlib ~
Machine learning linear regression
I tried deep learning
Machine learning course memo
Machine learning library dlib
Machine learning (TensorFlow) + Lotto 6
Somehow learn machine learning
Machine learning library Shogun
Machine learning rabbit challenge
Introduction to machine learning
Machine Learning: k-Nearest Neighbors
What is machine learning?
I tried to move machine learning (ObjectDetection) with TouchDesigner
Machine learning beginners try to make a decision tree
I tried to compress the image using machine learning
Everything for beginners to be able to do machine learning
Machine learning model considering maintainability
Machine learning learned with Pokemon
Data set for machine learning
Japanese preprocessing for machine learning
Uncle SE with hardened brain tried to study machine learning
Learning flow for Python beginners
An introduction to machine learning
Machine learning / classification related techniques
Machine Learning: Supervised --Linear Regression
Basics of Machine Learning (Notes)
Python beginners publish web applications using machine learning [Part 1] Introduction
[Machine learning] Understanding random forest
Machine learning with Python! Preparation
Machine Learning Study Resource Notepad
Machine learning ② Naive Bayes Summary
I tried using Tensorboard, a visualization tool for machine learning
Understand machine learning ~ ridge regression ~.
I tried machine learning to convert sentences into XX style
Machine learning article summary (self-authored)
About machine learning mixed matrices
Machine Learning: Supervised --Random Forest
Machine learning beginners try to reach out to Naive Bayes (2) --Implementation
Practical machine learning system memo
Machine learning Minesweeper with PyTorch