Notify Slack of how Keras is learning

image.png

Recently, I often train models with Python's deep learning library Keras. Deep learning generally takes a long time to train the model, and I don't always log in to the GPU (or similar powerful) server to check the console, so I would appreciate it if you could report the progress each time. is.

So, from Keras' callback function, hit Slack's WebAPI to try to notify by chat.

Get Slack WebAPI Key

First, get the required API key. The task of getting an API key for Slack is just a summary, as there are a lot of resources out there and the official documentation is kind.

From Slack Official, press the "Start Building" button at the top of the screen. Then, the name of the app that uses the API key (appropriate and OK, for example, "notifier") and the pop-up that registers the team that wants to use the API key will appear. Enter it as appropriate. image.png

Once the app is registered, we want to post to Slack externally this time, so enable Incoming Webhooks. 貼り付けた画像_2017_04_26_14_49.png

After that, if you follow the instructions on the screen, the API key will be issued. 貼り付けた画像_2017_04_26_14_51.png

When the API key is issued, the sample code will be displayed, so it is a good idea to copy and paste it to check the operation once. 貼り付けた画像_2017_04_26_14_53.png

Keras callback settings

Keras provides a class called LambdaCallback that makes it easy to generate custom callbacks, so you can just use it:

import os

from keras.callbacks import LambdaCallback

hostname = os.uname()[1]

callbacks = []

slack_command = 'curl -X POST -H \'Content-type: application/json\' --data \'{{"text":"Here is {}.\nepoch:{:03d}, loss:{:.7f}, val_loss:{:.7f}"}}\' https://hooks.slack.com/services/<your_key_here>'
slack_callback = LambdaCallback(
    on_epoch_end=lambda epoch, logs: os.system(slack_command.format(hostname, epoch, logs['loss'], logs['val_loss'])))
callbacks.append(slack_callback)

#All you have to do is pass a Callback when calling the Keras model training method as usual.
history = model.fit_generator(train_generator, samples_per_epoch, nb_epoch, callbacks=callbacks,
                    validation_data=val_generator, nb_val_samples=nb_val_samples)

Replace <your_key_here> with your API key. Also, loss and val_loss exist without anything, but if you want other metrics, you can pass them to the metrics argument of themodel.compile ()method and name themLambdaCallback.You can also refer to it. For registration of custom metrics, refer to Applicable page of Keras official document.

As you can see from the code above, LambdaCallback has hooks such as ʻon_epoch_begin and ʻon_batch_end in addition to ʻon_epoch_end`. However, it would be annoying if Slack was notified on a batch-by-batch basis, so I might not use it for this purpose. (It may be convenient for logging to CSV etc.)

Reference link

Recommended Posts

Notify Slack of how Keras is learning
Regularly notify Slack of missed Backlog issues
Notify Slack when the machine learning process running on GCP is finished
I tried to notify slack of Redmine update
Parallel learning of deep learning by Keras and Kubernetes
Notify Slack of process state changes using Supervisor's EventListener
Basics of Python learning ~ What is a string literal? ~
Record of the first machine learning challenge with Keras
Summary of Tensorflow / Keras
Deep learning 1 Practice of deep learning
What is ensemble learning?
What is machine learning?
[TensorFlow 2 / Keras] How to run learning with CTC Loss in Keras
[Reinforcement learning] Explanation and implementation of Ape-X in Keras (failure)
How to increase the number of machine learning dataset images
A beginner's summary of Python machine learning is super concise.