[TF] How to use Tensorboard from Keras

keras is a wrapper library of ** Theano / Tensorflow **, and by using ** keras **, less code than writing solid in Theano / Tensorflow It can be described by quantity. Regarding keras, there is a page that has already been explained, so I will give it to that, and I investigated how to use Tensorboard from keras, so I would like to explain it briefly.

When outputting the log for ** Tensorboard **, use the mechanism called ** Callback **. The registered ** Callback ** will be called at a fixed timing. The timing of being called depends on the type of ** Callback **, but ** Callback ** for ** Tensorboard ** is called ** just before learning ** and ** at the end of each epoch **. Will be done.

Just before learning, ** tf.merge_all_summaries ** and ** tf.train.SummaryWriter ** are called. At the end of each epoch, ** add_summary ** is called and a log is output. You can also change the frequency of calling callbacks if each epoch is too much.

** Callback ** for ** Tensorflow ** is created with ** keras.callbacks.TensorBoard **. Pass the created Callback to fit. In ** histogram_freq **, specify the frequency to output the histogram data of ** Tensorboard **. If histogram_freq = 1, epoch data will be output every time.

python


    tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, histogram_freq=1)
    cbks = [tb_cb]
            
    history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch = nb_epoch, verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))

If the network structure changes between ** Training ** and ** Test ** like ** Dropout **, the following description is required before building the Model. If you do not describe it, you will die when building the Model.

python


    KTF.set_learning_phase(1)

** learning_phase ** is used when ** Training ** and ** Test ** behave differently. If the network does not change between ** Training ** and ** Test **, it will not be entered.

** learning_phase ** is set to ** 1 ** during ** Training ** and ** 0 ** during ** Test **. This value switches the network. You don't have to do anything because ** keras ** automatically inputs the value. For example, ** 0 ** is set for ** Validation **.

Simple example

A simple ** Maltilayer Neural Network ** example looks like this:

python


import numpy as np
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.optimizers import SGD
from keras.utils import np_utils
import keras.callbacks
import keras.backend.tensorflow_backend as KTF
import tensorflow as tf

batch_size = 128
nb_classes = 10
nb_epoch   = 20
nb_data    = 28*28
log_filepath = './log'

# load data
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# reshape
X_train = X_train.reshape(X_train.shape[0], X_train.shape[1]*X_train.shape[2])
X_test = X_test.reshape(X_test.shape[0], X_test.shape[1]*X_test.shape[2])

# rescale
X_train = X_train.astype(np.float32)
X_test  = X_test.astype(np.float32)
X_train /= 255
X_test  /= 255

# convert class vectors to binary class matrices (one hot vectors)
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

old_session = KTF.get_session()

with tf.Graph().as_default():
    session = tf.Session('')
    KTF.set_session(session)
    KTF.set_learning_phase(1)
    # build model
    model = Sequential()
    model.add(Dense(512, input_shape=(nb_data,), init='normal',name='dense1'))
    model.add(Activation('relu', name='relu1'))
    model.add(Dropout(0.2, name='dropout1'))
    model.add(Dense(512, init='normal', name='dense2'))
    model.add(Activation('relu', name='relu2'))
    model.add(Dropout(0.2, name='dropout2'))
    model.add(Dense(10, init='normal', name='dense3'))
    model.add(Activation('softmax', name='softmax1'))       
    model.summary()

    model.compile(loss='categorical_crossentropy', optimizer=SGD(lr=0.01), metrics=['accuracy'])

    tb_cb = keras.callbacks.TensorBoard(log_dir=log_filepath, histogram_freq=1)
    cbks = [tb_cb]
            
    history = model.fit(X_train, Y_train, batch_size=batch_size, nb_epoch = nb_epoch, verbose=1, callbacks=cbks, validation_data=(X_test, Y_test))

    score = model.evaluate(X_test, Y_test, verbose=0)
    print('Test score:', score[0])
    print('Test accuracy;', score[1])

KTF.set_session(old_session)

Doing this will print the ** Keras ** runtime log to standard output.

Layer (type)                       Output Shape        Param #     Connected to                     
====================================================================================================
dense1 (Dense)                     (None, 512)         401920      dense_input_1[0][0]              
____________________________________________________________________________________________________
relu1 (Activation)                 (None, 512)         0           dense1[0][0]                     
____________________________________________________________________________________________________
dropout1 (Dropout)                 (None, 512)         0           relu1[0][0]                      
____________________________________________________________________________________________________
dense2 (Dense)                     (None, 512)         262656      dropout1[0][0]                   
____________________________________________________________________________________________________
relu2 (Activation)                 (None, 512)         0           dense2[0][0]                     
____________________________________________________________________________________________________
dropout2 (Dropout)                 (None, 512)         0           relu2[0][0]                      
____________________________________________________________________________________________________
dense3 (Dense)                     (None, 10)          5130        dropout2[0][0]                   
____________________________________________________________________________________________________
softmax1 (Activation)              (None, 10)          0           dense3[0][0]                     
====================================================================================================
Total params: 669706
____________________________________________________________________________________________________
Train on 60000 samples, validate on 10000 samples
Epoch 1/20
60000/60000 [==============================] - 5s - loss: 1.2522 - acc: 0.6646 - val_loss: 0.5603 - val_acc: 0.8695
Epoch 2/20
60000/60000 [==============================] - 5s - loss: 0.5533 - acc: 0.8445 - val_loss: 0.3862 - val_acc: 0.8962
Epoch 3/20
60000/60000 [==============================] - 5s - loss: 0.4381 - acc: 0.8728 - val_loss: 0.3291 - val_acc: 0.9084
Epoch 4/20
60000/60000 [==============================] - 5s - loss: 0.3879 - acc: 0.8867 - val_loss: 0.2963 - val_acc: 0.9147
Epoch 5/20
60000/60000 [==============================] - 6s - loss: 0.3536 - acc: 0.8975 - val_loss: 0.2753 - val_acc: 0.9198
Epoch 6/20
60000/60000 [==============================] - 5s - loss: 0.3271 - acc: 0.9047 - val_loss: 0.2575 - val_acc: 0.9268
Epoch 7/20
60000/60000 [==============================] - 5s - loss: 0.3059 - acc: 0.9113 - val_loss: 0.2421 - val_acc: 0.9301
Epoch 8/20
60000/60000 [==============================] - 5s - loss: 0.2873 - acc: 0.9168 - val_loss: 0.2302 - val_acc: 0.9331
Epoch 9/20
60000/60000 [==============================] - 5s - loss: 0.2751 - acc: 0.9199 - val_loss: 0.2198 - val_acc: 0.9359
Epoch 10/20
60000/60000 [==============================] - 5s - loss: 0.2612 - acc: 0.9246 - val_loss: 0.2092 - val_acc: 0.9386
Epoch 11/20
60000/60000 [==============================] - 5s - loss: 0.2507 - acc: 0.9274 - val_loss: 0.1998 - val_acc: 0.9416
Epoch 12/20
60000/60000 [==============================] - 6s - loss: 0.2383 - acc: 0.9311 - val_loss: 0.1930 - val_acc: 0.9437
Epoch 13/20
60000/60000 [==============================] - 5s - loss: 0.2288 - acc: 0.9340 - val_loss: 0.1859 - val_acc: 0.9461
Epoch 14/20
60000/60000 [==============================] - 5s - loss: 0.2195 - acc: 0.9367 - val_loss: 0.1776 - val_acc: 0.9492
Epoch 15/20
60000/60000 [==============================] - 5s - loss: 0.2107 - acc: 0.9391 - val_loss: 0.1715 - val_acc: 0.9511
Epoch 16/20
60000/60000 [==============================] - 5s - loss: 0.2040 - acc: 0.9405 - val_loss: 0.1658 - val_acc: 0.9514
Epoch 17/20
60000/60000 [==============================] - 5s - loss: 0.1969 - acc: 0.9423 - val_loss: 0.1607 - val_acc: 0.9533
Epoch 18/20
60000/60000 [==============================] - 5s - loss: 0.1922 - acc: 0.9442 - val_loss: 0.1559 - val_acc: 0.9554
Epoch 19/20
60000/60000 [==============================] - 5s - loss: 0.1853 - acc: 0.9454 - val_loss: 0.1518 - val_acc: 0.9558
Epoch 20/20
60000/60000 [==============================] - 6s - loss: 0.1813 - acc: 0.9470 - val_loss: 0.1472 - val_acc: 0.9568
('Test score:', 0.1472099754229188)
('Test accuracy;', 0.95679999999999998)

Looking at the log output by ** Tensorboard **, it looks like the following.

image

The histograms of ** Weight **, ** Bias **, and ** Activation ** are output as shown below. The Histogram Tag is a bit like that, it seems that the name is not used for now.

image

Graph looks like this.

image

I was able to easily output the ** Tensorboard ** log from ** Keras **. ** Keras ** has just been released and there is still room for improvement, but it seems to be updated daily. The description is cleaner than writing ** Tensorflow ** in solid, so I think it's very convenient when you want to give it a try.

Recommended Posts

[TF] How to use Tensorboard from Keras
How to use Keras ~ From simple model generation to CNN ~
How to use SWIG from waf
How to use xml.etree.ElementTree
How to use Python-shell
How to use tf.data
How to use virtualenv
How to use Seaboan
How to use image-match
How to use shogun
How to use Pandas 2
How to use numpy.vectorize
How to use pytest_report_header
How to use partial
How to use Bio.Phylo
How to use SymPy
How to use x-means
How to use WikiExtractor.py
How to use IPython
How to use virtualenv
How to use Matplotlib
How to use iptables
How to use numpy
How to use TokyoTechFes2015
How to use venv
How to use dictionary {}
How to use Pyenv
How to use list []
How to use python-kabusapi
How to use OptParse
How to use return
How to use dotenv
How to use pyenv-virtualenv
How to use Go.mod
How to use imutils
How to use import
Study from Python Hour7: How to use classes
How to use Qt Designer
How to use search sorted
[gensim] How to use Doc2Vec
python3: How to use bottle (2)
How to use the generator
[Python] How to use list 1
How to use Azure Table storage from Django (PTVS)
A memorandum on how to use keras.preprocessing.image in Keras
How to use Python argparse
How to use IPython Notebook
How to use Pandas Rolling
[Note] How to use virtualenv
How to use redis-py Dictionaries
Python: How to use pydub
[Python] How to use checkio
[Go] How to use "... (3 periods)"
How to use Django's GeoIp2
[Python] How to use input ()
How to use the decorator
[Introduction] How to use open3d
How to use Python lambda
How to use Jupyter Notebook
[Python] How to use virtualenv
python3: How to use bottle (3)