Use scikit-learn training dataset with chainer (for learning / prediction)

Note that the np.array dataset used in Scikit-learn could not be used immediately in Chainer (1.11).

To prepare

--Training dataset: train_X, train_y --Test dataset: test_X, test_y

here

--train_X and test_X are np.array of (number of samples, dimensions). Usually float32. --train_y and test_y are np.array of (number of samples,). Probably int64 by default.

For sklearn

SVM example with sklearn


from sklearn import svm
clf = svm.SVC()
clf.fit(train_X, train_y) #Learning
clf.score(test_X, test_y) #Forecast / score calculation
prob_X = clf.predict_proba(test_X) #Probability prediction

For Chainer

Use chainer trainer



#First, create a class.
class sk_dataset(chainer.dataset.DatasetMixin):
    def __init__(self, X, y):
        self.X = X.astype('float32') #Float32 is required, so convert here
        self.y = y.astype('int32')   #Since int32 is required, convert here
    def __len__(self):
        return self.X.shape[0]
    def get_example(self, i):
        return self.X[i], self.y[i]

# np.sk array_Convert to dataset type
train = sk_dataset(train_X, train_y)
test  = sk_dataset(test_X, test_y)

#The rest is tutorial street
model = L.Classifier(MLP())
optimizer = chainer.optimizers.SGD()
optimizer.setup(model)

train_iter = iterators.SerialIterator(train, batch_size=100)
test_iter  = iterators.SerialIterator(test,  batch_size=100, repeat=False, shuffle=False)

updater = training.StandardUpdater(train_iter, optimizer)
trainer = training.Trainer(updater, (20, 'epoch'), out='result')

trainer.run() #Learning


###Forecast
i = 10 #i-th data
dim = 4096 #Data dimension

print("predict:", model.predictor(Variable(test[i][0].reshape((1,dim)))).data) #Raw output

print("probability:", F.softmax(model.predictor(Variable(test[i][0].reshape((1,dim)))).data).data) #Probability prediction

print("label:", F.softmax(model.predictor(Variable(test[i][0].reshape((1,dim)))).data).data.argmax()) #Label prediction


Recommended Posts

Use scikit-learn training dataset with chainer (for learning / prediction)
Use tensorboard with Chainer
Create a dataset of images to use for learning
Use chainer with Jetson TK1
Try horse racing prediction with Chainer
[Chainer] Learning XOR with multi-layer perceptron
Try machine learning with scikit-learn SVM
Python learning memo for machine learning by Chainer Chapter 9 Introduction to scikit-learn
Try Common Representation Learning with chainer
Python learning memo for machine learning by Chainer Chapter 13 Neural network training ~ Chainer completed
scikit-learn How to use summary (machine learning)
Use DeepL with python (for dissertation translation)
Amplify images for machine learning with python
Classify anime faces with deep learning with Chainer
Try with Chainer Deep Q Learning --Launch
[Python] Use string data with scikit-learn SVM
[Shakyo] Encounter with Python for machine learning
Array buffer object for use with Cython
I tried to implement various methods for machine learning (prediction model) using scikit-learn.
Python learning notes for machine learning with Chainer Chapters 11 and 12 Introduction to Pandas Matplotlib
Creating learning data for face image dataset sorting (# 1)
Wrap C with Cython for use from Python
[Python] Collect images with Icrawler for machine learning [1000 images]
Machine learning Training data division and learning / prediction / verification
Wrap C ++ with Cython for use from Python
I installed Chainer, a framework for deep learning