Loading page…
Pitch
Description
Comments
Project
slackker
Monitor Python scripts & ML training in real-time on Slack & Telegram.
0likes0comments
Project details
About this project
slackker is a Python package designed to simplify the monitoring of scripts and ML model training. Integrate it easily with any Python function, receive real-time updates, and gain insights while on the go. Customize the metrics you track, export training plots, and reclaim your time without sacrificing oversight.
slackker is a Python package designed for monitoring the training status of Python scripts and machine learning models in real-time via Slack and Telegram. It alleviates the hassle of frequently checking the training metrics, allowing for more efficient use of time when running models.
Key Features
- Seamless Integration: Easily integrate slackker within any Python script or function.
- Real-Time Notifications: Receive instant updates on training progress directly to Slack and Telegram.
- Visualizations: Automatically export training metrics plots and send them to designated Slack channels.
- Custom Alerts: Tailor the metrics to track and notifications sent according to your needs.
- User-Friendly: Simple setup—just import the package and configure Slack or Telegram, and it’s ready to go.
With slackker, the need to constantly monitor the training process diminishes. Users can engage in other activities, such as grabbing coffee or running errands, while still receiving timely notifications about model performance, keeping peace of mind.
Setup slackker
- Slack: [How to setup slackker for your slack channel][setup-slack]
- Telegram: [How to setup slackker for Telegram][setup-telegram]
Use slackker callbacks for any python functions
Import basic slackker callbacks with following line:
from slackker.callbacks.basic import SlackUpdate # for slack
###################### OR ######################
from slackker.callbacks.basic import TelegramUpdate # for telegram
Create slackker object.
# for Slack
slackker = SlackUpdate(token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
channel="A04AAB77ABC")
or
# for Telegram
slackker = TelegramUpdate(token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG")
token: (string) Slack app/Telegram tokenchannel: (string) Slack channel where you want to receive updatesverbose: (int) default0: You can sent the verbose level up to 3.verbose = 0No loggingverbose = 1Info loggingverbose = 2Debug/In-depth logging
Now you can wrap your function with this slackker object.
@slackker.notifier
def your_function():
return value_1, value_2
following messages will be sent to your slack channel when the function executes.
Function 'your_function' from Script: 'your_script.py' executed.
Execution time: 5.006 Seconds
Returned 2 outputs:
Output 0:
value_1
Output 1:
value_2
You can also use slackker.notify(*args, **kwargs) at the end of your script to notify the end of script execution.
if __name__ == "__main__":
your_function()
slackker.notify(arg1, f"This is argument 2 = {arg2}", value="This is a string")
following message will be sent to your slack channel when the script ends.
Your script: 'your_script.py' has been executed successfully at 14-10-2024 12:15:54
arg1
This is argument 2 = arg2
value: This is a string
Final code for python function
from slackker.callbacks.basic import SlackUpdate
###################### OR ######################
from slackker.callbacks.basic import TelegramUpdate # for telegram
# for Slack
slackker = SlackUpdate(token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
channel="A04AAB77ABC")
###################### OR ######################
# for Telegram
slackker = TelegramUpdate(token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG")
@slackker.notifier
def your_function():
return value_1, value_2
slackker.notify(f"This is value 1: {value_1}", value=value_2)
Use slackker callbacks with [Keras][keras]
Import slackker for Keras
Import slackker callbacks for keras with following line:
from slackker.callbacks.keras import SlackUpdate # for slack
###################### OR ######################
from slackker.callbacks.keras import TelegramUpdate # for telegram
Create slackker object for keras
Create slackker object.
# for Slack
slackker = SlackUpdate(token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
channel="A04AAB77ABC",
ModelName='Keras_NN',
export='png',
SendPlot=True)
or
# for Telegram
slackker = TelegramUpdate(token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG",
ModelName='Simple_NN',
export='png',
SendPlot=True)
token: (string) Slack app/Telegram tokenchannel: (string) Slack channel where you want to receive updatesModelName: (string) Name for your model. This same name will be used in future for title of the generated plots.export: (string) default"png": Format for plots to be exported. (supported formats: eps, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff)SendPlots: (Bool) defaultFalse: If set toTrueit will export history of model, both training and validation, save it in the format given inexportargument and send graphs to slack channel when training ends. If set toFalseit will not send exported graphs to slack channel.verbose: (int) default0: You can sent the verbose level up to 3.verbose = 0No loggingverbose = 1Info loggingverbose = 2Debug/In-depth logging
Call slackker object into model.fit()
Now you can call slackker object into callbacks argument just like any other callbacks object.
history = model.fit(x_train,
y_train,
epochs = 3,
batch_size = 16,
verbose=1,
validation_data=(x_val,y_val),
callbacks=[slackker])
Final code for Keras
# Import library for keras
from slackker.callbacks.keras import slackUpdate
# Train-Test split for your keras model
x_train, x_test, y_train, y_test = train_test_split(x, y, train_size=0.8)
x_train, x_val, y_train, y_val = train_test_split(x_train, y_train, train_size=0.8)
# Build keras model
model = Sequential()
model.add(Dense(8,activation='relu',input_shape = (IMG_WIDTH, IMG_HEIGHT, DEPTH)))
model.add(Dense(3,activation='softmax'))
model.compile(optimizer = 'rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
# Create Slackker object
slackker = slackUpdate(token="xoxb-123234234235-123234234235-adedce74748c3844747aed48499bb",
channel="A04AAB77ABC",
modelName='SampleModel',
export='png',
sendPlot=True)
# Call Slackker object in model.fit() callbacks
history = model.fit(x_train,
y_train,
epochs = 3,
batch_size = 16,
verbose=1,
validation_data=(x_val,y_val),
callbacks=[slackker])
Use slackker callbacks with [Lightning][lightning]
Import slackker for Lightning
Import slackker callbacks for PyTorch Lightning with following line:
from slackker.callbacks.lightning import SlackUpdate # for slack
###################### OR ######################
from slackker.callbacks.lightning import TelegramUpdate # for telegram
Log your metrics to track
Log Training loop metrics
self.log("train_loss", loss, on_epoch=True)
self.log("train_acc", accuracy, on_epoch=True)
Make sure to set on_epoch=True to in training step.
Log Validation loop metrics
self.log("val_loss", loss)
self.log("val_acc", accuracy)
In Validation step on_epoch=True by default.
Create slackker object for lightning
# for Slack
slackker = SlackUpdate(token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
channel="A04AAB77ABC",
ModelName='Lightning NN',
TrackLogs=['train_loss', 'train_acc', 'val_loss', 'val_acc'],
monitor="val_loss",
export='png',
SendPlot=True)
or
# for Telegram
slackker = TelegramUpdate(token="1234567890:AAAAA_A111BBBBBCCC2DD3eEe44f5GGGgGG",
ModelName="Lightning NN Testing",
TrackLogs=['train_loss', 'train_acc', 'val_loss', 'val_acc'],
monitor="val_loss",
export='png',
SendPlot=True)
token: (string) Slack app/Telegram tokenchannel: (string) Slack channel where you want to receive updatesModelName: (string) Name for your model. This same name will be used in future for title of the generated plots.TrackLogs: (list) List of metrics you want slackker to track & notify.monitor: (string) This metric will be used to determine best Epochexport: (string) default"png": Format for plots to be exported. (supported formats: eps, jpeg, jpg, pdf, pgf, png, ps, raw, rgba, svg, svgz, tif, tiff)SendPlots: (Bool) defaultFalse: If set toTrueit will export history of model, both training and validation, save it in the format given inexportargument and send graphs to slack channel when training ends. If set toFalseit will not send exported graphs to slack channel.verbose: (int) default0: You can sent the verbose level up to 3.verbose = 0No loggingverbose = 1Info loggingverbose = 2Debug/In-depth logging
Call slackker object in Trainer module
Now you can call slackker object into callbacks argument just like any other callbacks object.
trainer = Trainer(max_epochs=2,callbacks=[slackker])
Final code for Lightning
import torch
import torch.nn as nn
from torch.utils.data import DataLoader
import torchvision as tv
import torch.nn.functional as F
from lightning.pytorch import LightningModule, Trainer
from lightning.pytorch.callbacks import ModelCheckpoint, Callback
from lightning.pytorch.loggers import CSVLogger
from slackker.callbacks.lightning import SlackUpdate
from slackker.callbacks.lightning import TelegramUpdate
class LightningModel(LightningModule):
def __init__(self):
super().__init__()
self.fc1 = nn.Linear(28*28,256)
self.fc2 = nn.Linear(256,128)
self.out = nn.Linear(128,10)
def forward(self, x):
batch_size, _, _, _ = x.size()
x = x.view(batch_size,-1)
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
return self.out(x)
def configure_optimizers(self):
optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)
return optimizer
def training_step(self, batch, batch_idx):
x, y = batch
y_hat = self.forward(x)
# calculate Loss
loss = F.cross_entropy(y_hat,y)
#calculate accuracy
_, predictions = torch.max(y_hat, dim=1)
correct_predictions = torch.sum(predictions == y)
accuracy = correct_predictions / y.shape[0]
self.log("train_loss", loss, on_epoch=True)
self.log("train_acc", accuracy, on_epoch=True)
return loss
def validation_step(self, batch, batch_idx):
x, y = batch
y_hat = self.forward(x)
# calculate Loss
loss = F.cross_entropy(y_hat,y)
#calculate accuracy
_, predictions = torch.max(y_hat, dim=1)
correct_predictions = torch.sum(predictions == y)
accuracy = correct_predictions / y.shape[0]
self.log("val_loss", loss)
self.log("val_acc", accuracy)
return loss
train_data = tv.datasets.MNIST(".", train=True, download=True, transform=tv.transforms.ToTensor())
test_data = tv.datasets.MNIST(".", train=False, download=True, transform=tv.transforms.ToTensor())
train_loader = DataLoader(train_data, batch_size=128)
test_loader = DataLoader(test_data, batch_size=128)
model = LightningModel()
# slackker checkpoint for slack
slackker = SlackUpdate(token="xoxb-123234234235-123234234235-adedce74748c3844747aed",
channel="A04AAB77ABC",
ModelName='Lightning NN',
TrackLogs=['train_loss', 'train_acc', 'val_loss', 'val_acc'],
monitor="val_loss",
export='png',
SendPlot=True)
trainer = Trainer(max_epochs=2, callbacks=[slackker])
trainer.fit(model, train_loader, test_loader)
Comments
0Start the conversation
Share the first comment.