I decided to start Keras in earnest, so I wrote the code to build the prediction model of XOR. I'd like to create a complex model such as Graph Convolutional Networks in the future, so I'll start with the Functional API.
Like this. Without Batch Normalization, it was easy to fall into a local solution, so I added it. In many cases, the final layer was a linear layer, but it was a 0, 1 classification problem and I was not convinced, so I chose the sigmoid function. The number of units in the two intermediate layers is set to eight.
sample.py
import tensorflow as tf
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Dense, Input, Dropout, BatchNormalization
import numpy as np
def main():
x_input = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y_input = np.array([[0], [1], [1], [0]])
input_tensor = Input(shape=(x_input.shape[1]))
# x = Dense(units=32, activation="tanh", kernel_initializer='random_normal')(input_tensor)
x = Dense(units=8, activation="relu", kernel_initializer='random_normal', use_bias=True)(input_tensor)
x = BatchNormalization()(x)
x = Dropout(0.1)(x)
x = Dense(units=8, activation="relu", kernel_initializer='random_normal', use_bias=True)(x)
kernel_initializer='random_normal', use_bias=False)(x)
output_layer = Dense(units=1, activation='sigmoid', kernel_initializer='random_normal', use_bias=False)(x)
model = Model(input_tensor, output_layer)
model.compile(loss='mse', optimizer='sgd', metrics=['accuracy'])
model.summary()
#Learning
model.fit(x_input, y_input, nb_epoch=2000, batch_size=2, verbose=2)
#Forecast
print(model.predict(np.array([[0, 0]])))
print(model.predict(np.array([[1, 0]])))
print(model.predict(np.array([[0, 1]])))
print(model.predict(np.array([[1, 1]])))
if __name__ == "__main__":
main()
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 2)] 0
_________________________________________________________________
dense (Dense) (None, 8) 24
_________________________________________________________________
batch_normalization (BatchNo (None, 8) 32
_________________________________________________________________
dropout (Dropout) (None, 8) 0
_________________________________________________________________
dense_1 (Dense) (None, 8) 72
_________________________________________________________________
dense_2 (Dense) (None, 1) 8
=================================================================
Total params: 136
Trainable params: 120
Non-trainable params: 16
[[0.09092495]]
[[0.9356866]]
[[0.90092343]]
[[0.08152929]]
I tried it as a starting point this time, but I would like to try various things such as parameter tuning and visualization.
Recommended Posts