[TensorFlow] [Keras] Neural network construction with Keras

In this article, "Build a neural network with Python without using a library --Qiita" An article (Neural network construction with chainer --Qiita) that a senior of the company tried with Chainer I also tried it with Keras (material for LT in the company)

Keras Documentation

All sources and execution results (Jupyter Notebook)

The source and execution result are listed in the following Gist. Building a neural network with Keras

Implementation and description

Data creation

Create input data. This is exactly the same implementation.

Input data (cum-teacher data) creation


import numpy as np
import sklearn.datasets
import matplotlib
import matplotlib.pyplot as plt

np.random.seed(0)
X,y=sklearn.datasets.make_moons(200,noise=0.20)
plt.scatter(X[:,0], X[:,1], s=40, c=y, cmap=plt.cm.Spectral)

ダウンロード.png

Modeling

Create a model in Keras.

Modeling


import keras
from keras.models import Sequential
from keras.layers import Dense, Activation

model = Sequential()
model.add(Dense(output_dim=6, input_dim=2))
model.add(Activation('tanh'))
model.add(Dense(output_dim=2))
model.compile(optimizer='Adam', loss='mse')

It's about the same as I ported Chainer's. (However, it seems that the loss function is hidden in Classifier in Chainer, but what is used?) => [Addition] In the following article, it was said that softmax_cross_entropy is used in Chainer's Classifier. Notes on changes in Chainer 1.5 --studylog / North Cloud

Type Set value
Input layer 2
Hidden layer 6
Activation function tanh
Output layer 2
Optimizer (optimization algorithm) Adam
Objective function (loss function) Mean squared error(Mean Squared Error)

Sequential Model Guide-Keras Documentation Activation function --Keras Documentation Optimization-Keras Documentation Objective Function-Keras Documentation

(Epoch was 20000 in Chainer, but it was mistakenly changed to 2000 at the time of transplantation, and it was a poor result if the hidden layer was 3 which is the same as Chainer. When the hidden layer was set to 6, it was 2000. I got a good result)

Learning

Train with model.fit (). Regarding y_train, it is converted into a two-dimensional array (vector) with a probability of 0 and a probability of 1. (Chainer seems to convert it without permission)

x_train = X
y_train = keras.utils.np_utils.to_categorical(y, nb_classes=2)
model.fit(x=x_train, y=y_train, nb_epoch=2000)

Numpy Utility --Keras Documentation

Result display

Result output


# https://gist.github.com/dennybritz/ff8e7c2954dd47a4ce5f
def plot_decision_boundary(pred_func):
    # Set min and max values and give it some padding
    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5
    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5
    h = 0.01
    # Generate a grid of points with distance h between them
    xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
    # Predict the function value for the whole gid
    Z = pred_func(np.c_[xx.ravel(), yy.ravel()])
    Z = Z.reshape(xx.shape)
    # Plot the contour and training examples
    plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)
    plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Spectral)

def predict(model, x_data):
    y = model.predict(x_data)
    return np.argmax(y.data, axis=1) #Get the index that is the maximum value

plot_decision_boundary(lambda x: predict(model, x))

ダウンロード (1).png

I think this gives almost the same result as the original article.

Recommended Posts

[TensorFlow] [Keras] Neural network construction with Keras
Neural network with Python (scikit-learn)
Neural network starting with Chainer
4. Circle parameters with neural network!
TensorFlow Tutorial-Convolutional Neural Network (Translation)
MNIST (DCNN) with Keras (TensorFlow backend)
Neural network with OpenCV 3 and Python 3
Simple classification model with neural network
Compose with a neural network! Run Magenta
Predict time series data with neural network
Compare raw TensorFlow with tf.contrib.learn and Keras
Persist the neural network built with PyBrain
2. Mean and standard deviation with neural network!
Build a classifier with a handwriting recognition rate of 99.2% with a TensorFlow convolutional neural network
Zundokokiyoshi with TensorFlow
Breakout with Tensorflow
Parametric Neural Network
Verification of Batch Normalization with multi-layer neural network
Summary of Tensorflow / Keras
Train MNIST data with a neural network in PyTorch
Image recognition with keras
[Environment construction] @anaconda that runs keras / tensorflow on GPU
Reading data with TensorFlow
Kyotei forecast with TensorFlow
Implement Convolutional Neural Network
Identify the name from the flower image with keras (tensorflow)
CIFAR-10 tutorial with Keras
Implement Neural Network from 1
Convolutional neural network experience
Challenge image classification with TensorFlow2 + Keras 3 ~ Visualize MNIST data ~
Multivariate LSTM with Keras
I tried to implement Grad-CAM with keras and tensorflow
Learn Wasserstein GAN with Keras model and TensorFlow optimization
Try regression with TensorFlow
I ran the TensorFlow tutorial with comments (first neural network: the beginning of the classification problem)
I tried a convolutional neural network (CNN) with a tutorial on TensorFlow on Cloud9-Classification of handwritten images-
Create a web application that recognizes numbers with a neural network
2020/02 Python 3.7 + TensorFlow 2.1 + Keras 2.3.1 + YOLOv3 Object detection with the latest version
Try to build a deep learning / neural network with scratch
Python sample to learn XOR with genetic algorithm with neural network
[TensorFlow 2 / Keras] How to run learning with CTC Loss in Keras
[Deep learning] Image classification with convolutional neural network [DW day 4]
Construction of a neural network that reproduces XOR by Z3
[Ubuntu 18.04] Tensorflow 2.0.0-GPU environment construction
Implement a 3-layer neural network
Translate Getting Started With TensorFlow
Simulate neural activity with Brian2
Try deep learning with TensorFlow
Use TensorFlow with Intellij IDEA
Multiple regression analysis with Keras
Approximate sin function with TensorFlow
Auto Encodder notes with Keras
Implemented word2vec with Theano + Keras
Sentence generation with GRU (keras)
Tuning Keras parameters with Keras Tuner
ML environment construction with Miniconda
Jetson Nano JETPACK 44.1 (2020/10/21) with Tensorflow
Stock price forecast with tensorflow
Python environment construction and TensorFlow
Easily build CNN with Keras
Neural network implementation in python