Keras learns the weights of variables, but there are also fixed parameters such as how many neurons are good, how many layers are needed, and the optimal learning rate. This time, I will pass it to GridSearch of scikit-learn to tune these parameters. In this example, it is written assuming the case of classifying with data such as iris.
In addition to the libraries needed to build the model, load the following:
python
from sklearn.grid_search import GridSearchCV
from sklearn.pipeline import Pipeline
Create a function with the parameter you want to change as an argument. In this example, for the sake of simplicity, the number of neurons in the second and subsequent layers is calculated as the nth root of the original number of neurons.
python
def create_model(self, evMethod, neurons, layers, learn_rate):
# Criate model
model = Sequential()
model.add(Dense(neurons, input_dim=self.column_length, kernel_initializer='normal', activation='relu'))
for i in range(1, layers):
model.add(Dense(int(numpy.ceil(numpy.power(neurons,1/i))), kernel_initializer='normal', activation='relu'))
model.add(Dense(3, kernel_initializer='normal', activation='softmax'))
# Compile model
adam = optimizers.Adam(lr=learn_rate)
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy'])
return model
All must be passed as an array.
python
batch_size = [1000,300,100]
epochs = [10, 50, 100]
neurons = [20, 30, 45]
learn_rate = [0.001, 0.005, 0.01, 0.07]
layers = [1,3,5]
Add the created model as mlp
to the estimator. The initial value is not used, so it is appropriate.
python
estimators = []
estimators.append(('mlp', KerasClassifier(build_fn=self.create_model, epochs=10, batch_size=200, verbose=1)))
pipeline = Pipeline(estimators)
python
param_grid = dict(mlp__neurons = neurons, mlp__batch_size = batch_size, mlp__epochs=epochs, mlp__learn_rate=learn_rate, mlp__layers=layers)
python
grid = GridSearchCV(estimator=pipeline, param_grid=param_grid)
grid_result = grid.fit(self.X, self.Y)
best_estimator_
will tell you the parameters of the best model. The accuracy is also displayed.
python
clf = grid_result.best_estimator_
print(clf.get_params())
accuracy = clf.score(self.X, self.Y)
print("\nAccuracy: %.2f" % (accuracy))
Using GridSearch takes dozens of times longer learning time, so it is also recommended to save the best model. Please note that the call is special compared to the case of normal model construction.
python
clf.steps[1][1].model.save('file to path')
You can tune other variables such as activation function in this way, so please take advantage of it.
Since the memory was over during learning, I changed from just GridSearch to Randomized GridSearch to reduce the number of trials. Almost all you have to do is replace the function name. If you have a large number of parameter combinations, this is more realistic for execution.
python
grid = RandomizedSearchCV(estimator=pipeline, param_distributions=param_grid, n_iter=30)
Recommended Posts