View progress status when backup is running through cron job

remss · October 27, 2017, 9:41am

Hi,

How to view process status when a backup command is running through a cron job ?
I currently redirect stdout/stderr to a log file but progress is not present.

Thanks!

wscott · October 27, 2017, 11:25am

I have been doing backups on Linux and use lsof(1) to get an idea of what a backup is doing in the background. Often when a backup is taking much longer than expected, I find I forgot to exclude some big directory that changes all the time. This tells me what is being backed up right now, but doesn’t give any kind of progress other than estimating how far into the filesystem we are.

I am still toying with the idea of writing a friendly frontend to restic and it would be nice to have some API to connect to an existing backup and see what is going on currently in order to generate status bars on a GUI.

fd0 · October 28, 2017, 10:55am

Progress is only printed when stdout/stderr is a terminal, which is not the case for a cron job. There’s no way yet to force printing progress information without a terminal.

remss · October 30, 2017, 8:40am

Ok thank you.
Is a such feature planed in the roadmap ?

alphapapa · January 30, 2018, 6:53am

@remss Have you checked the issue tracker? You should open an issue if necessary.

rawtaz · January 30, 2018, 12:34pm

Not the same but related/similar: https://github.com/restic/restic/issues/1427

Other than that, there was a discussion on the IRC (Freenode, #restic) a while back, where I suggested a KISS approach where one is able to specify an output file for progress information, that restic regularly updates when it’s doing work.

The suggested file would contain a representation of the % of completion. It could do that at 1% intervals, that would mean at most 100 writes. One could make it even less (configurable such as --progress-file path[:%intervals]).

This would be a very generic solution that could be used by anyone and anything, and it could even be made to contain not only a number for the % complete, but something like: {RUNNING|OK|FAIL}:<PERCENTAGE>:<TIMESTAMP_STARTED>[:ERROR_MESSAGE]

alphapapa · January 31, 2018, 10:00pm

That’s a good idea. Since Restic often has an accurate count of how much work to do (e.g. since it does a full scan before starting a backup), I’d suggest outputting raw numbers rather than just percentages. Another idea is to output JSON objects, one per line, with absolute counts and percentages, at certain intervals.

luken · February 8, 2018, 4:44pm

I am using the lsof method as well, here is a script that automates the process and weeds out most of the uninteresting files.

# cat restic_open_files.sh 
#!/bin/bash

rpid=$(ps ax -o fname,pid | sed -e '/restic /!d;s/.* //')
echo $rpid
lsof -p $rpid -n | grep REG | sed -e '/tmp.restic-/d'

iluvcapra · October 31, 2018, 3:02am

Just checking in on this, I’m running Restic on macOS in a Platypus wrapper, and in that situation stdout and stderr aren’t a terminal and there’s basically no way to get a status report I can tell.

matt · October 31, 2018, 3:16am

Follow this PR: https://github.com/restic/restic/pull/1944

And probably this one too: https://github.com/restic/restic/pull/2003

mic_p · November 5, 2018, 9:01pm

Hi,
we submit a PR at https://github.com/restic/restic/pull/1905 but it hasn’t been accepted.

We are using it in our env and it works perfectly without stdout + stderr.

If you need some other infos or help, feel free to contact me.

Bye

mgj · August 20, 2019, 2:28pm

You can get around this by running the command over ssh with the “-t” option (force pseudo-tty allocation) to localhost (obviously once you’ve got keys setup) and then pipe that output to a log file

e.g.

{ssh -t localhost “restic -v backup …”} > log.txt 2>&1

Do bear in mind that these logs can get quite big fast (the log for for a single day’s backup of my servers is over 100MB)

pwr · August 27, 2019, 6:00am

launchd is finicky and pedantic, but pretty good for running on OS X with output to a log file.

I use BitBar and wrote the Launch Agents plugin to get GUI control.

cweilguny · October 31, 2020, 4:33pm

Hi forum folks,

is there an update to the progress and status topic?

I used restic only to backup small amounts of data until now and it worked all the time i used and needed it. Now I also do backup a volume containing about 1TB of data, and the troubles begin. Broken backups, stopped backups, no way to determine how long it will take (1TB over 20Mbps upload to a remote host is more than a blink of an eye). As long as it works (or you believe it works, you can’t know until you do a full restore and check the content) it’s OK. When troubles occour, I’m not confident about it anymore. No need to discuss or solve my troubles here, I just wanted to try to mention some of the problems that appear. I really would like to see features, that make it easier to monitor running backups. Using lsof you can at least see which files are backuped NOW, but that’s useless if you’re backing up 1.000.000 files. The whole process needs logs and tools to see what’s up with the backup. I really love restic, so no hate, but lsof, ps or redirecting output of the backup command are poor workarounds. I don’t do backups just for fun. A backup you can’t trust is like a backup you don’t test and that is like a backup that doesn’t exist.

So I hope there is some progress on this (he, get that ambiguous pun ?). I’m not demanding anything, just want to give my oppinion how important this is for backups.

Cheers

Aunt Edit says: And I’m also not screaming for a GUI, that’s another job. Just proper ways to monitor a running backup process.