I implemented Extreme learning machine

What is Extreme Learning Machine (ELM)?

ELM is a special form of feedforward perceptron. It has one hidden layer, but the weight of the hidden layer is randomly determined, and the weight of the output layer is determined using the pseudo inverse matrix. In terms of image, the hidden layer is like making a lot of random feature extractors and selecting features in the output layer.

ELM has the following characteristics.

Implementation

import numpy as np


class ExtremeLearningMachine(object):
    def __init__(self, n_unit, activation=None):
        self._activation = self._sig if activation is None else activation
        self._n_unit = n_unit

    @staticmethod
    def _sig(x):
        return 1. / (1 + np.exp(-x))

    @staticmethod
    def _add_bias(x):
        return np.hstack((x, np.ones((x.shape[0], 1))))

    def fit(self, X, y):
        self.W0 = np.random.random((X.shape[1], self._n_unit))
        z = self._add_bias(self._activation(X.dot(self.W0)))
        self.W1 = np.linalg.lstsq(z, y)[0]

    def transform(self, X):
        if not hasattr(self, 'W0'):
            raise UnboundLocalError('must fit before transform')
        z = self._add_bias(self._activation(X.dot(self.W0)))
        return z.dot(self.W1)

    def fit_transform(self, X, y):
        self.W0 = np.random.random((X.shape[1], self._n_unit))
        z = self._add_bias(self._activation(X.dot(self.W0)))
        self.W1 = np.linalg.lstsq(z, y)[0]
        return z.dot(self.W1)

test

I will try it with iris for the time being.

Method

from sklearn import datasets

iris = datasets.load_iris()
ind = np.random.permutation(len(iris.data))

y = np.zeros((len(iris.target), 3))
y[np.arange(len(y)), iris.target] = 1

acc_train = []
acc_test = []
N = [5, 10, 15, 20, 30, 40, 80, 160]
for n in N:
    elm = ExtremeLearningMachine(n)
    elm.fit(iris.data[ind[:100]], y[ind[:100]])
    acc_train.append(np.average(np.argmax(elm.transform(iris.data[ind[:100]]), axis=1) == iris.target[ind[:100]]))
    acc_test.append(np.average(np.argmax(elm.transform(iris.data[ind[100:]]), axis=1) == iris.target[ind[100:]]))
plt.plot(N, acc_train, c='red', label='train')
plt.plot(N, acc_test, c='blue', label='test')
plt.legend(loc=1)
plt.savefig("result.png ")

result

result.png

Conclusion

Recommended Posts

I implemented Extreme learning machine
Machine learning
I tried machine learning with liblinear
[Reinforcement learning] I implemented / explained R2D3 (Keras-RL)
I installed Python 3.5.1 to study machine learning
I implemented CycleGAN (1)
[Memo] Machine learning
Machine learning classification
I implemented ResNet!
Machine Learning sample
I started machine learning with Python Data preprocessing
Machine learning tutorial summary
About machine learning overfitting
Machine learning ⑤ AdaBoost Summary
Machine Learning: Supervised --AdaBoost
Machine learning logistic regression
Machine learning support vector machine
Studying Machine Learning ~ matplotlib ~
I implemented Python Logging
I tried deep learning
Machine learning course memo
Machine learning library dlib
Machine learning (TensorFlow) + Lotto 6
Somehow learn machine learning
Machine learning library Shogun
Machine learning rabbit challenge
Introduction to machine learning
Machine Learning: k-Nearest Neighbors
What is machine learning?
What I learned about AI / machine learning using Python (1)
[Deep Learning from scratch] I implemented the Affine layer
I tried to move machine learning (ObjectDetection) with TouchDesigner
What I learned about AI / machine learning using Python (3)
[Python] I made a classifier for irises [Machine learning]
What I learned about AI / machine learning using Python (2)
I tried to compress the image using machine learning
Machine learning model considering maintainability
Machine learning learned with Pokemon
Data set for machine learning
Japanese preprocessing for machine learning
Machine learning in Delemas (practice)
An introduction to machine learning
Machine Learning: Supervised --Linear Regression
Basics of Machine Learning (Notes)
I started machine learning with Python Clustering & Dimension Compression & Visualization
Machine learning beginners tried RBM
I installed the automatic machine learning library auto-sklearn on centos7
[Machine learning] Understanding random forest
What I learned about AI and machine learning using Python (4)
Machine learning with Python! Preparation
Machine Learning Study Resource Notepad
I implemented VQE with Blueqat
I tried using Tensorboard, a visualization tool for machine learning
Understand machine learning ~ ridge regression ~.
I tried machine learning to convert sentences into XX style
Machine learning article summary (self-authored)
About machine learning mixed matrices
Machine Learning: Supervised --Random Forest
Practical machine learning system memo
Machine learning Minesweeper with PyTorch
Machine learning environment construction macbook 2021