Alerting & Monitoring

Getting Help
1

Hi All,

ive just moved to restic in docker and am looking for a good way to alert and monitor backups, what is everyone else doing?

1 Like
2

I tried out a bunch of different tools and eventually landed on the heartbeat model, specifically Healthchecks. You can either host it yourself or use their hosted version. Basically, you send a signal when your backup runs. If it doesn’t finish on time, you get an alert through-I think-about 30 different methods. You can also handle errors in your script and send a failure signal manually. It makes monitoring backups way more efficient and takes the pressure off the admin to check logs constantly.

`echo “>>> [1/3] Backing up Node-RED…”
/usr/local/bin/restic backup 
“/var/lib/docker/volumes/node_red_node-red-data/_data” 
–tag “node-red”

if [ $? -ne 0 ]; then
echo “!!! FAILURE: Node-RED backup failed”
GLOBAL_ERROR=1
fi
if [ $GLOBAL_ERROR -eq 0 ]; then
curl -fsS -m 10 --retry 3 “$HC_URL” > /dev/null
else
curl -fsS -m 10 --retry 3 “$HC_URL/fail” > /dev/null
exit 1
fi`
3

@bolto90 i second that; i also use https://healthchecks.io/ with notifications. I think it is great.
Next to that I use dashboards in Grafana, you can find some examples for that here:

  1. rest-server
    rest-server/examples/compose-with-grafana at master · restic/rest-server · GitHub

  2. healthchecks
    Configuring Prometheus - Healthchecks.io

  3. npbackup dashboard
    GitHub - netinvent/npbackup: A secure and efficient file backup solution that fits both system administrators (CLI) and end users (GUI) · GitHub

To be honest, dashboards are nice to get visual/statistical insights, but alerting is more important for me. Nothing as bad as a scheduled backup that does not run or aborts due to disk full / broken network connection / unreadible source / …