View progress status when backup is running through cron job


#1

Hi,

How to view process status when a backup command is running through a cron job ?
I currently redirect stdout/stderr to a log file but progress is not present.

Thanks!


#2

I have been doing backups on Linux and use lsof(1) to get an idea of what a backup is doing in the background. Often when a backup is taking much longer than expected, I find I forgot to exclude some big directory that changes all the time. This tells me what is being backed up right now, but doesn’t give any kind of progress other than estimating how far into the filesystem we are.

I am still toying with the idea of writing a friendly frontend to restic and it would be nice to have some API to connect to an existing backup and see what is going on currently in order to generate status bars on a GUI.


#3

Progress is only printed when stdout/stderr is a terminal, which is not the case for a cron job. There’s no way yet to force printing progress information without a terminal.


#4

Ok thank you.
Is a such feature planed in the roadmap ?


#5

@remss Have you checked the issue tracker? You should open an issue if necessary.


#6

Not the same but related/similar: https://github.com/restic/restic/issues/1427

Other than that, there was a discussion on the IRC (Freenode, #restic) a while back, where I suggested a KISS approach where one is able to specify an output file for progress information, that restic regularly updates when it’s doing work.

The suggested file would contain a representation of the % of completion. It could do that at 1% intervals, that would mean at most 100 writes. One could make it even less (configurable such as --progress-file path[:%intervals]).

This would be a very generic solution that could be used by anyone and anything, and it could even be made to contain not only a number for the % complete, but something like: {RUNNING|OK|FAIL}:<PERCENTAGE>:<TIMESTAMP_STARTED>[:ERROR_MESSAGE]


#7

That’s a good idea. Since Restic often has an accurate count of how much work to do (e.g. since it does a full scan before starting a backup), I’d suggest outputting raw numbers rather than just percentages. Another idea is to output JSON objects, one per line, with absolute counts and percentages, at certain intervals.


#8

I am using the lsof method as well, here is a script that automates the process and weeds out most of the uninteresting files.

# cat restic_open_files.sh 
#!/bin/bash

rpid=$(ps ax -o fname,pid | sed -e '/restic /!d;s/.* //')
echo $rpid
lsof -p $rpid -n | grep REG | sed -e '/tmp.restic-/d'

#9

Just checking in on this, I’m running Restic on macOS in a Platypus wrapper, and in that situation stdout and stderr aren’t a terminal and there’s basically no way to get a status report I can tell.


#10

Follow this PR: https://github.com/restic/restic/pull/1944

And probably this one too: https://github.com/restic/restic/pull/2003


#11

Hi,
we submit a PR at https://github.com/restic/restic/pull/1905 but it hasn’t been accepted.

We are using it in our env and it works perfectly without stdout + stderr.

If you need some other infos or help, feel free to contact me.

Bye