Data Soup
Sometimes I want to check the progress of my model without connecting to the server. Here you can make your model log its progress at each epoch into a Slack channel. Apart from Keras, it doesn’t require any installation and should be done in 5min.
The objective is to make a lightweight tool but we can imagine uploading accuracy/loss graph
on on_train_end and much more if needed!
Getting a result similar to the image above, can be done in three short steps: create the bot, import the callback and attach it to fit.
Create a bot and get the webhook
- Create an app, name it and assign it to your workspace
- Activate
Incoming Webhooks, set the desired channel - Notice the
webhook, it looks like:https://hooks.slack.com/services/...
More information on bots.
Import the callback
(...)
from keras.callbacks import Callback
class SlackLogger(Callback):
"""Callback that streams epoch results to a csv file.
Supports all values that can be represented as a string,
including 1D iterables such as np.ndarray.
# Example
```python
slacklog = SlackLogger('WEBHOOKSTRING')
model.fit(X_train, Y_train, callbacks=[slacklog])
```
# Arguments
webhook: webhook of your bot (https://api.slack.com/bot-users#installing-bot for more infos).
separator: string used to separate elements each slack message
experiment_name: name of the experimentation (will be announced when train begin)
# SlackLogger is an adaptation of the CSVLogger class
"""
(...)
def send_message(self, message):
command = ["curl", "-X", "POST", "-H", "'Content-type: application/json'",
"--data", '{{"text":"{}"}}'.format(message), self.webhook]
subprocess.call(command)
def on_train_begin(self, logs=None):
self.send_message('⏱️ {} training started'.format(self.experiment_name))
self.start_time = time.time()
def on_epoch_end(self, epoch, logs=None):
logs = logs or {}
(...)
row_dict = OrderedDict({})
row_dict.update((key, handle_value(logs[key])) for key in self.keys)
perfs = self.sep.join([" {}: {:.5f}".format(key, value) for key, value in row_dict.items()])
message = '#{} {}'.format(epoch, perfs)
self.send_message(message)
def on_train_end(self, logs=None):
elapsed_time = int(time.time() - start_time)
styled_time = '{:03d}:{:02d}:{:02d}'.format(elapsed_time // 3600, (elapsed_time % 3600 // 60), elapsed_time % 60)
self.send_message('🏁 {} training done in {}'.format(self.experiment_name, styled_time))
Attach the callback to the fit_* method
hook = 'YOUR_WEBHOOK'
experiment_name = 'Foo Bar classification'
slacklog = SlackLogger(webhook=hook, experiment_name=experiment_name)
model.(..., callbacks=[slacklog])