Try implementing XOR with Keras Functional API

Introduction

I decided to start Keras in earnest, so I wrote the code to build the prediction model of XOR. I'd like to create a complex model such as Graph Convolutional Networks in the future, so I'll start with the Functional API.

Source

Like this. Without Batch Normalization, it was easy to fall into a local solution, so I added it. In many cases, the final layer was a linear layer, but it was a 0, 1 classification problem and I was not convinced, so I chose the sigmoid function. The number of units in the two intermediate layers is set to eight.

sample.py


import tensorflow as tf
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Dense, Input, Dropout, BatchNormalization

import numpy as np


def main():
    x_input = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
    y_input = np.array([[0], [1], [1], [0]])

    input_tensor = Input(shape=(x_input.shape[1]))
    # x = Dense(units=32, activation="tanh", kernel_initializer='random_normal')(input_tensor)
    x = Dense(units=8, activation="relu", kernel_initializer='random_normal', use_bias=True)(input_tensor)
    x = BatchNormalization()(x)
    x = Dropout(0.1)(x)
    x = Dense(units=8, activation="relu", kernel_initializer='random_normal', use_bias=True)(x)
kernel_initializer='random_normal', use_bias=False)(x)
    output_layer = Dense(units=1, activation='sigmoid', kernel_initializer='random_normal', use_bias=False)(x)
    model = Model(input_tensor, output_layer)

    model.compile(loss='mse',  optimizer='sgd', metrics=['accuracy'])
    model.summary()

    #Learning
    model.fit(x_input, y_input, nb_epoch=2000, batch_size=2, verbose=2)

    #Forecast
    print(model.predict(np.array([[0, 0]])))
    print(model.predict(np.array([[1, 0]])))
    print(model.predict(np.array([[0, 1]])))
    print(model.predict(np.array([[1, 1]])))


if __name__ == "__main__":
    main()

Model overview

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         [(None, 2)]               0
_________________________________________________________________
dense (Dense)                (None, 8)                 24
_________________________________________________________________
batch_normalization (BatchNo (None, 8)                 32
_________________________________________________________________
dropout (Dropout)            (None, 8)                 0
_________________________________________________________________
dense_1 (Dense)              (None, 8)                 72
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 8
=================================================================
Total params: 136
Trainable params: 120
Non-trainable params: 16

Prediction result

[[0.09092495]]
[[0.9356866]]
[[0.90092343]]
[[0.08152929]]

in conclusion

I tried it as a starting point this time, but I would like to try various things such as parameter tuning and visualization.

Recommended Posts

Try implementing XOR with Keras Functional API
Try implementing XOR with PyTorch
Try implementing RBM with chainer.
Try implementing perfume with Go
Multi-input / multi-output model with Functional API
Beginner RNN (LSTM) | Try with Keras
Try using Dropbox API v2 with Go
Try implementing a Cisco Spark bot with AWS Lambda + Amazon API Gateway (Python)
Try slack OAuth authentication with flask (Slack API V2)
[AWS] Try tracing API Gateway + Lambda with X-Ray
[First API] Try to get Qiita articles with Python
Pre-processing to build seq2seq model using keras Functional API
Try implementing associative memory with Hopfield network in Python
Try MNIST with VAT (Virtual Adversarial Training) in Keras
Try to make RESTful API with MVC using Flask 1.0.2
Building a seq2seq model using keras' Functional API Inference
Try implementing MetaTrader's LWMA with scipy's FIR filter function
Try scraping with Python.
CIFAR-10 tutorial with Keras
Try SNN with BindsNET
Extrude with Fusion360 API
Multivariate LSTM with Keras
Try NNabla's C ++ API
Try regression with TensorFlow
Try implementing gRPC structured logs easily and simply with grpc_zap
Try hitting the Twitter API quickly and easily with Python
CNN with keras Try it with the image you picked up
(For beginners) Try creating a simple web API with Django
Image analysis with Object Detection API to try in 1 hour